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Introduction 


by 

K. S. Fu 

Purdue University 

The NASA Workshop on Image Analysis held on April 28-30, 1982 at 
Texas A&M University, College Station, Texas, provided an opportunity for 
experts in the areas of pattern recognition, image processing, and remote 
sensing to assess past progress and to project future development in the 
area of image analysis with respect to remote sensing applications. 

A block diagram of the general image analysis system is given in 
Figure 1. The preprocessing stage usually refers to filtering, enhance- 
ment, and/or coding of raw imagery data. The segmentation stage involves 
the determination of various regions of importance in the image. Features 
such as shape and texture measurement are then extracted from each region; 
a classification technique is often employed to recognize these regions. 
Once each region has been recognized and the relations among these 
regions have been identified, a complete description and possibly the 
interpretation of the image can be obtained through a structural 
analysis. A priori knowledge (the so-called "world model") of the 
images under study plays an important role in the design of each stage. 

The program of the three-day workshop was devoted to the three major 
topics of image analysis: segmentation, shape and texture analysis, and 
structural analysis. A survey paper and two or three special papers 
were presented on each topic. Formal presentations were followed by 
panel discussions which assessed past progress and identified future 
research problems in each topic area. 
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Image Description (and Interpretation) 


Block diagram of image analysis system 
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Robert M. Haralick, /irginia Polytechnic and State University 

11:30 - 

12:30 

Coope»^tive Processes in Iniuge Segmentation 
Larry Davis, University of Maryland 

12:30 - 

2:00 

Lunch 

2:00 - 

3:00 

Dual Problems in Image Segmentation 
Jack Bryant, Texas A&M University 
Susan Jenson, EROS Data Center 

3:00 - 

3:30 

Coffee Break 

3-30 - 

5:00 

Panel Discussion-- Image Segmentation 
Moderator: Azriel Rosenfeld 
Panelists: R. K. Aggarwal 
Jack Bryant 
Larry Davis 
Robert M. Haralick 


Thursday, April 29 : 

8:15 - 8:30 Coffee and donuts 

8:30 - 9:30 Shape and Texture 

Azriel Rosenfeld, University of Maryland 


PRECEDING PAGE BLANK NOT FILMED 


NASA WORKSHOP ON IMAGE ANALYSIS, conf d. 


9:30 - 

10:30 

Shape Identification Using 3-D Features 
Carolyn M. Bjorklund, Lockheed Palo Alto 

10:30 - 

11:00 

Coffee Break 

11:00 - 

12:00 

Automatic Photointerpretation Via Textural Feature 
Extraction 

Julius P. Tou, University of Florida 

12:00 - 

1:30 

Lunch 

1:30 - 

2:30 

Target Screening 

R. K. Aggarwal, Honeywell 

2:30 - 

3:00 

Coffee Break 

3:00 - 

4:00 

Panel 0iscussion--Shape and Texture 
Moderator: Robert M. Haralick 
Panelists: Carolyn M. Bjorklund 
Azriel Rosenfeld 
Julius Tou 

Friday, 

April 30: 


8:15 - 

8:30 

Coffee and Donuts 

8:30 - 

9:30 

Structure Analysis Techniques for Remote Sensing 
Linda Shapiro, Virginia Polytechnic and State University 

9:30 - 

10:30 

Determining 3-D Motion and Structure from Image Sequences 
Thomas S. Huang, University of Illinois 

10:30 - 

11:00 

Coffee Break 

11:00 - 

12:30 

Panel Discussion--Image Structure Analysis 
Moderator: Thomas S. Huang 
Panelists: K. S. Fu 


Linda Shapiro 


5 


NASA WORKSHOP ON IMAGE ANALYSIS 
April 28-30, 1982 


Participants and Other Attendees ; 


Raj Aggarwal 
Honeywell, MNl 7-2349 
Systems and Research Center 
2600 Ridgeway Parkway 
Minneapolis, MN 55413 

Martha L. Amis (C31) 

Lockheed E.M.S.C.O. 

1830 NASA Road 1 
Houston, TX 77258 

Carolyn M. Bjorklund 
Lockheed Palo Alto Research Labs 
3251 Hanover Street, Dept. 52-53 
Palo Alto, CA 94304 

Jack Bryant 

Department of Mathematics 
Texas A&M University 
College Station, TX 77843 

William A. Coberly 

Division of Mathematical Sciences 

University of Tulsa 

Tulsa, OK 74104 

Larry Davis 

Computer Science Center 
University of Maryland 
College Park, MD 20742 

K. S. Fu 

School of Electrical Engineering 

Purdue University 

West Lafayette, IN 47907 

Larry F. Guseman, Jr. 

Department of Mathematics 
Texas A&M University 
College Station, TX 77843 


Robert M. Haralick 

Department of Electrical Engineering 
Virginia Polytechnic and State Univ. 
Blacksburg, VA 24061 

Richard P. Heydorn/SG3 
NASA/Johnson Space Center 
Earth Observation Division 
Houston, Texas 77058 

Thomas S. Huang 

Coordinated Science Labor: tory 
University of Illinois 
1101 W. Springfield Avenue 
Urbana, IL 61801 

Susan Jenson 
EROS Data Center 
Applications Branch 
Sioux Falls, SD 57198 

Michael Merickel 
Lockheed E.M.S.C.O. 

1830 NASA Road 1 
Mail Code C31 
Houston, TX 77258 

R. S. Rajaram/C31 
Lockheed 
1830 NASA Road 1 
Houston, TX 77258 

Azriel Rosenfield 
Computer Science Center 
University of Maryland 
College Park, MD 20742 

Linda G. Shapiro 
Department of Computer Science 
Virginia Polytechnic and State Univ. 
Blacksburg, VA 24061 


6 


NASA Workshop on Image Analysis 
Participants and Attendees, cont'd. 


Larry L. Schumaker 

Department of Mathematics and Center for 
Approximation Theory 
Texas A&M dnlversity 
College Station, TX 77843 

Julius T. Tou 

Center for Information Research 
University of Florida 
Gainesville, FL 32611 

Curtis E. Woodcock 
Department of Geography 
University of California 
Santa Barbara, CA 93106 



7 


tV8S 15767 


Image Segmentation Survey 


Robet N. Haralick 

Virginia Polytechnic Institute and State University 
Dept, of Electrical Engineering 
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Blacksburgr VA 24061 


Image segmentation can be accomplished by a variety 
techniques which in this survey we classify as: 

Single linkage schemes 
Hybrid linkage schemes 
Centroid linkage schemes 
Histogram Mode Seeking Schemes 
Spatial Clusterning schemes 
Split and Merge Schemes 
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SXnale Llnlcage ImAflfi Seantentation 

Single linkage image segmentation schemes regard each pixel 
as a node in a graph. Neighboring pixels whose properties are 
similar enough are joined by an arc. The image segments are 
maximal sets of pixels all belonging to the same connected 
component. Single linkage image segmentation schemes are 
attractive for their simplicity. They do, however, have a problem 
with chaining, because it takes only one arc leaking from one 
region to a neighboring one to cause the regions to merge. 

The simplest single linkage scheme defines similar enough by 
pixel difference. Two neighboring pixels are similar enough if 
the absolute value of the difference between their gray tone 
intensity value is small enough. For pixels having vector values, 
the obvious generalization is to use a vector norm of the pixel 
difference vector. Instead of using a Euclidean distance, Asano 
and Yokoya (1981) suggest that two pixels be joined together if 
this absolute value of their difference is small enough compared 
to the average absolute value of the center pixel minus neighbor 
pixel for each of the neighborhoods the pixels belong to. 
Haralick and Dinstein (1975) , however, do report some success 
using the simpler Euclidean distance on LANDSAT data. The ease 
with which unwanted region chaining can occur with this technique 
limits its potential on complex or noisy data. 

Hybrid single linkage techniques are more powerful than the 
simple single linkage technique. The hybrid techniques seek to 
assign a property vector to each pixel where the property vector 
depends on the KxK neighborhood of the pixel. Pixels which are 




sirailarr are similar because their neighborhoods in some special 


sense are similar. Similarity is thus established as a f:unction 
of neighboring pixel values and this makes the technique better 
behaved on noisy data. 

One hybrid single linkage scheme relies oh an edge operator 
to establish whether two pixels are joined with an arc. Here an 
edge operator is applied to the image labeling each pixel as edge 
or non-edge. Neighboring pixels^ neither of which are edgesr are 


joined by an arc. 


The initial segments ?re the connected 


components of the non-edge labeled pixels. The edge pixels can 
either be left assigned edges and be considered as background or 
they can be assigned to the spatially nearest region having a 
label. 

The quality of this technique is highly dependent on th<£ edge 
operator used. Simple operators such as the Roberts and Sobel 
operator may provide too much region linkage^ for a region cannot 
be declared as a segment unless it is completely surrounded by 
edge pixels. Haralick (1982) reports some success with thio 
technique using the zero-crossing of second directional derivative 
edge operator. 

Another hybrid technique first used by Levine and Leemet 
(1976) is based on the Jarvis and Patrick (1973) shared nearest 
neighbor idea. Using any kind of reasonable notion for 
similarity» each pixel examines its KxK neighborhood and makes a 
list of the N pixels in the neighborhood most similar to it. Call 
this list the similar neighbor listr where we understand neighbor 
to be any pixel in the KxK neighborhood. An arc joins any pair of 
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immediately neighboring pixels if there are enough pixels common 
to their shared neighbor lists; that is, if the number of shared 
neighbors is high enough. 

To make the shared neighbor technique work well each pixel 
can be associated with a property vector conitisting of its own 
gray tone intensity and a suitable average of the gray tone 
intensity of pixels in its KxK neighborhood. For example, we can 
have (x,a) and (y,b) denote the property vectors for two pixels 
where in the first pixel, x is its gray tone intensity value and a 
is the average gray tone intensity value in its neighborhood. 
Likewise, for the second pixel, y is its gray tone intensity value 
and b is the average gray tone intensity value in its 
neighborhood. Similarity can be established by computing 

S = Wj^(x-y)^ + W2(x-b)^ + 

where w^, W2 and w^ are non-negative weights. The pixels are 
called similar enough for small enough values of s. 

Region Growing L Centroid XlL’ age 

In contrast to single linkage, in centroid linkage pairs of 
neighboring pixels are not compared for similarity. Rather, a 
pixel's value is compared to the centroid of an already existing 
but not necessarily completed segment. If the values are close 
enough, then the pixel is added to the segment and the segment's 
centroid is updated. If no neighboring region has a centroid 
close enough, >ien a new segment is established having the given 
pixel's value as its first member. Such a region growing 
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technique was first suggested by Brice and Finnema (1970). 
Instead of using the absolute value of the difference as the 
measure of dis-similarity, Gupta^ Kettig, Landgrebe, and Wintz 
(1973) suggest using the more appropriate t-test. 

Simple single pass approaches which scan the image in a left 
right top down manner are, of course, unable to make the left and 
right sides of a V-shaped region belong to the same segment. To 
be more effective, the single pass must be followed by some kind 
of connected components algorithm in which pairs of neighboring 
regions having centroids which are close enough are put into the 
same segment. 

One minor problem with centroid linkage schemes is their 
inherent dependence on the order in which pixels are examined. A 
left right top down scan does not yield the same initial regions 
as a right left bottom up scan or for that matter a column major 
scan. Usually, however, differences caused by scan order are 
minor. 

Histogram Seeking 

Histogram mode seeking is a measurement space clustering 
process in which the clusters in measurement space are mapped back 
to the image domain where the maximal connected components of the 
clusters constitute the image segments. For images which are 
single band images, calculation of this histogram in an array is 
direct. The measurement space clustering can be accomplished by 
determining the valleys in this histogram and declaring the 
clusters to be the interval of values between valleys. A pixel 
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whose value is in the i^^ interval is labeled with index i and the 
segment it belongs to is a connected component of all pixels whose 
label is i. 

For multiband images such as LANDSAT, determining the 

histogram in a multi-dimensional array is n6t feasible. For 

examplOf in a six band image where each band has intensitites 

6 12 

between 0 and 99 r the array would have to have 100 = 10 

locations. A large image might be 10,000 pixels per row by 10,000 

D 

rows. This only constitutes 10 pixels, a sample too small to 

12 

estimate probabilities in a space of 10 values were it not for 
some constraints of reality: (1) there is typically a high 

correlation between the band to band pixel values and (2) there is 
a large amount of spatial redundancy in image data. Both these 

O 

factors create a situation in which the 10 pixels can be expected 

4 5 

to contain only between 10 and 10 distinct 6-tuples. Based on 
this fact, the counting required for the histogram is easily done 
by hashing the 6-tuple into an array. 

Clustering using the multidimensional histogram is more 
difficult than univariate histogram clustering. Goldberg and 
Shlien (1977, 1978) threshold the multidimensional histogram to 

select all N-tuples situated on the most prominent modes. Then 
they perform a measurement space connected components on these N- 
tuples to collect together all the N-tuples in the top of the most 
prominent modes. These measurement space connected sets form the 
cluster cores. The clusters are defined as the set of all N- 
tuples closest to each cluster core. A variation on this idea is 
discussed by Matsumoto, Naka, and Yamamoto (1981) 
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An alternate possibility is to locate the highest mode and 
region gcrw around it in the multi-dimensional measurement space. 
The region growing includes all successive neighboring N-tuples 
whose probability is no higher than the N-tuple from which it is 
growing. This procedure identifies the most prominent mode and 
its associated mountain as the first cluster core. Then the same 
procedure is repeated on the remaining N-tuples until all multi- 
dimensional peaks and their associated cores have been accounted 
for. The clusters are defined as the set of all N-tuples closest 
to each core. 

Rather than accomplish the clustering in the fulx measurement 
spacer it is possible to work in multiple lower order projection 
spaces and than reflect these clusters back to the full 
measurement space. Suppose, for example, that the clustering is 
done on a four band image. If the clustering done in bands 1 and 
2 yield clusters c^, C2, and the clustering done in bands 3 and 
4 yield clusters c^ and c^ than each possible 4-tuple from a pixel 
can be given a cluster label from the set {(c^,c^), (c^,c^) , 
(C2»c^), (c2fCj), (c^rC^), (CjrCg)}. A 4-tuple (x^,X2,X2,x^> gets 
the cluster label (C2,c^) if (x^,X2) is in cluster C2 and (x^rX^) 
is in cluster c^. 

It is possible to determine the image segments without 
performing an independent clustering in measurement space. Such 
techniques are called spatial clustering. In essence spatial 
clustering schemes combine the histogram mode seeking tecunique 
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with the region growing/centroid linkage technique. Haralick and 
Kelly (1969) suggested it be done by locatingr in turn, all the 
peaks in measurement space. Then determine all pixel locations 
ha\ ing a measurement on the peak. Beginning with a pixel 
corresponding to the highest peak not* yet processed, 
simultaneously perform a spatial and measurement space region 
growing in the following manner. Initially, each segment is the 
pixel from which we begin. Consider for possible inclusion into 
this segment the neighbors of this pixel (in general, the 
neighbors of the pixel we are growing from) if the neighbor's N- 
tuple value is close enough in measurement space to the pixel's 
value and if its probability is not larger than the probability of 
the pixel's value we are growing from. 

Split and Merge 

The split method for segmentation begins with the entire 
image as the initial segment. Then it successively splits each of 
its current segments into quarters if the socment is not 
homogeneous enough. Homogeneity can be easily established by 
determining if the difference between the largest and smallest 
gray tone intensities is small enough. Algorithms of this type 
were first suggested by Roberston (1973) and Klinger (1973) 

Because segments are successively divided into quarters the 
boundaries produced by the split technique tend to be squareish 
and slightly artificial. Sometimes adjacent quarters coming from 
adjacent split segments need to be joined rather than remain 
separate. Horowitz and Pavlidis (1976) suggest a split and merge 
strategy to take care of this problem Chen and Pavlidis (1980) 
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suggested using statistical tests for uniformity rather than 
examination of the difference between largest and smallest gray 
tone intensities. 

The data structures required to do a split and merge on 
images larger than 512x512 are extremely large.' Execution of the 
algorithm on virtual mememory computers results in so much paging 
that the dominant activity is paging rather than segmentation. 
Browning and Tanimoto (1982) give a description of a split and 
merge scheme where the split and merge first accomplished on 
mutually exclusive subimage blocks and the resulting segments are 
then merged between adjacent blocks to take care of tne artificial 
block boundaries. 
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pp. 267-273. 
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Cooperative processes In image segmentation 
Larry S. Davis 


This talk will survey recent research into the role 
of cooperative, or relaxation, processes in image segmenta- 
tion. Cooperative processes [1] can be employed at several 
levels of the segmentation process as a preprocessing en- 
hancement step [2,3,4], during supervised or unsupervised 
pixel classification [4,5] and, finally, for the interpreta- 
tion of image segments based on segment properties and rela- 
tions [6] . 

1. L. Davis and A. Rosenfeld, "Cooperative processes for 

low-level vision: A survey," A. I. 17 , 245-263, 1981. 

2. L. Davis and A. Rosenfeld, "Noise deeming by iterated 
local averaging," lEEET-SMC , 8 , 706-710, 1978. 

3. L. Davis and A. Mitiche, "MITES: A new tool for image 

segmentation," to appear in CGIP . 

4. K. Narayanan and A. Rosenfeld, "Image smoothing by local 
use of global information," lEEET-SMC , December 1981. 

5. L. Davis, C. Wang and H. Xie, "Some experiments in multi- 
spectral, multi-temporal crop classification using relaxa- 
tion," Univ. of Maryland Computer Science TR-1131, 

December 1981. 



6. L. Kitchen, "Scene analysis by region based constraint 
propagation," in preparation. 
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Cooperative Processes in Image Analysis 


Larry S, Davis 
Computer Vision Laboratory 
University of Maryland 
College Park, MD 207A2 
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Cooperative Processes 

Goal ; Assign symbolic and numerical labels to picture parts 

- ASSIGN SYMBOLIC LAND*USE CATEGORIES TO PIXELS 

- ASSIGN NUMERIC STEREO DISPARITY LABELS TO PIXELS 

Constraint ; Images are large , so labeling process must be fast 

- sequential; too slow 

- parallel; TOO error prone 
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2. Application to motion detection 
Assume 

1) Image intensity is a continuous^ differentiable 

FUNCTION F(X^Y^T) 

2) The intensity corresponding to any given scene 

POINT DOES NOT CHANGE OVER TIME 

3) Both the motion (u,v) and the time interval^ t, 

BETWEEN FRAMES IS SMALL ENOUGH THAT A IIaCLAUREN 
SERIES EXPANSION IS A GOOD LOCAL APPROXIMATION 
TO THE PICTUkE FUNCTION. 


Approximate fCx+u,y+v,t+t) by a series expansion about 
(XyY.Y) (which can be REGARDED AS (0>0^0)). 

F(x+UA+V,T+t) = f(X,Y,T)+F^'U+Fy'V+F^.t 

+hIGHER ORDER TERMS 

Arbitrarily set t=l and note that (2) implies f(x+u.y+v,t+t) 
fCx.y.t) 
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FxU+FyV 


: Motion constraint 


Fj • temporal derivative 

Fx^Fy * SPATIAL DERIVATIVES 


-Ft = FxU + FyV 


j) If Fx^Fy^U^ THEN MOTION INFORMATION CANNOT BE 
ACCURATELY DETERMINED 

2 ) If Fx=J> then -Fj=FyV so that v is determined but 

U IS UNKNOWN 

3) Let g anu ^ denote the gradient and level directions 
at a pixel. 

G = TAN"^ 
i 1 G^ F|l = 0 

Then f,. = f,. d^/Dt so component of velocity in the 

T u u I 

GRADIEiiT direction IS KNOWN/ BUT NOT IN THE LEVEL 
direction. 
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Combining Motion Constraints 


1. Assume that u.v are constant over small regions of the 

IMAGE AND OBTAIN A LOCAL LEAST SQUARE SOLUTION FOR U,V. 

C. Cafforio and F. Rocca, "Tracking moving objects in 
T.V. Images/' Signal Proc.. L 1979. 1]53-14U. 

J. Limb and J. Murphy. "Estimating the velocity of moving 
IMAGES IN T.V. SIGNALS." CGIP. 4 . 1975. 311-327. 

2. Assume that u.v vary smoothly over the image and use 

RELAXATION TECHNIQUES TO COMPUTE "oPTIMAL" VALUES AT U.V. 

b.K.P. Horn and B. Schunck. "Jetermining optical flow." 
ARTIFICIAL Intelligence. 17= 19ol. 185-203. 
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Supervised Relaxation 
- Statistical models for various textures 

M2 ... Mp 
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1) ProblX Interior to Mji - J problPj c Mjl • problx t Mjl 

2) ProbtX e Mj and on a vertical edge separating 

Mj from Mj 1 - 

TT ProblPj e Mjl •IT problPj e Mj). probtX e Mjl 
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Iterative procedure: 

For each plxeL X: ^ 

1) compute the most probable label for X 

2) Smooth X "appropriately" 


EX: 1) X Interior - average with all neighbors 



average X with 
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Noisy squares (a) and initial labelling (b) 




■ijf > 

iio^ 


• p" 

mIi ’* '• 

Mpl!^ 

llQPj 


R7 g9 ■ 












lyjg 


M ■ irii 

[MBKm' ff 














15th Iteration on Noisy Squares: 7x7 Simple Smoothing 
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TERATION ON NoiSY SQUARES: MITES 7 
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15th iteration smoothing (a) and 
labelling (b) . 3x3 simple smoothing 
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Figure 33 . 15th iteration smoothing (a) and labelling (b) -K 
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ON Contrast 
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Iteration Cluster on Contrast 
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HULTI-TEMPOP AL IMAGE SEGMENTATION 

Objects - pixels 

Labels - region class names 

Relaxation rules can be heuristic ally derived from the follow- 
ing OBSERVATION: 

Fields are> in ge e^al, much bigger than a single pixel and 
DO not change labels during a single growing season. 


PSf(l,J,T) - PROBABILITY THAT THE PIXEL AT SPATIAL LOCATION 
(I,J>T) and time T is IN class ^ AFTER K ITERA- 
TIONS OF THE RELAXATION RULE 



r 


ll 

i: 

li 

i: 

i: 

i: 

[ 

I 

I 
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OF POOR QUALITY 

Relaxation rules 

Rule schema 

^ Px(i.j,t) pJ(i>j,t') 

*F({pjJ. (l L.(r>J') A NEIGHBOR 
OF (l,j)}) 

Specific rules 
7 



7 2 

F2 = MAX ^ Px(Xj^.j,T) 
j =0 1=0 

Ft = I 

^ j = 0 . 2.^,6 


P^Xj.T) 



Tanksworld 




Scene Analysis - Tanks world 


Ground 

Sky 

Smoke 

Tank 

Tree 

Tree fragments 


Unary constraints 
Binary constraints 
Existential constraints 
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I Examples 

Binary constraints (arcs) 

Tank cannot surround smoke 

If (tank^smoke) « and if Nj surrounds Nj^ then 

DELETE (TANK^ smoke) FROM 

Existential constraints 

A TREE FRAGMENT MUST BE ADJACENT TO A TREE FRAGMENT 

j‘ If tree fragment « Lj and if for no adjacent Lj is 

(tree fragment, tree fragment) « THEN ELIMINATE 

[ TREE FRAGMENT FROM Lj . 

i: 
i: 
i 

r 

i 
[ 

C 

[ 

i; 

[ 
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COOrCRATIVl PIDCZSSES IM IIMCE ANALffll 


L. t. 

Co^t«T flalm Uboratory, Ce^tir Scl«BC« tetor, IhlvcrcUp «f 
CoUm* tmti., ItarpUiid 20742 


Ab« tract . ThU papct cootalaa aa evarrlaw of the eriaaiaatlaa of coeparatloo 
(ec lalaBatloo) pToceaaas fet loo-laval oliloa. Toe caaaples (aaa laoelrlai 
pUcl claaal/lcatlee mui the other ootlea 41aparltp oetlaatloo) era oead to 
Uloattate the varloua etepe Is applplap a relasetlea algarttha to laagc 
•Mlysis problflBS. 

Noto: Thii papar also appears in tJ»a Procoodia^a o* ITAC 12. 


1. ISSUES 

Itoej Im^c •cttlytl* probltat cm be regarded 
aa problaaa et acaigoiDg a label tP aacb 
alPMot la a ate ot picture pane (plaala 
or raglosa). For aseaplCp plaela caa be aa* 
algoad apaboUlc laad-^ae catagorp labala 
baaad on ttaair apactral algaacuraap or auBarl- 
cal ■DtiOB dlaparltp labala baaad oe local 
cooparlopoa aorw ; at cotxaacutlve fraoaa la a 
tloo-oarpiat ioaga. Borh of tbeaa preblaaa 
viU be uaad aa aaaplea ’throughout thla 
papax. 

Tba large auabar of plaela In a digital loage 
dMoada that awch labelllag proceaaaa be varp 
faat. One obvioua aolut:oo to thla probleo 
ia to oake the laballlng precaaa hlghlp 
parallal. Beuaver, In a parallel procaaa, 
aach picture part oould be aoalysod Indapan* 
dentlp of all other picture parta. Tkaio, 
parallel procaaaaa fall to make uaa of coo* 
tatuai laforaatioa (which la often avail* 
ahla)p and make nany laballlng errora. 

lo order to ovarcooe thla problm, one can 
aaaaoa the labelling poaalbUltlea for averp 
part iodapaodoDtlpp and than conpare aacb 
part'a aaaeaaoaata to thoae of otbarp related 
parta, la order to correct inconaiatonclaa. 
Since both tba aaacament aod the canparlaoo 
caa be done indepandaotlp for averj part, 
each ataga of tba proceaa ia parallal. On 
the other haodg contat la oov beli^ uaad at 
the cooparlaon ataga, vhae related parta are 
able to cewunlcata and 'cooperate *o To kaap 
the ceoputational coat low, the ecBparleooe 
ahould be local ; thap ahould iirvelve onlp 
parte tbat are diiectl* related (a.g., neigh- 
boring pUela). Thla localoeae can be con- 

fer bp iter et Ina tha conparlaon pro- 
ceaa, la order to allow infomatlon to pro- 
pagata. 


Tbeae conaideratioaa lead aaturallp to the 
dealga of a 'cooperative* approach to label* 
Xing picture parta which allow conteat to 
be uaad la the labelllag proceaa while otlll 
pemlttlag faat parallel iaplweotatloa and 
low cooputational coot. Such procaaaea are 
called 'ralaution* procaaaaa, bacauae of 
their raaeoblanca to car tala Iterdtlve pro- 
caaaeo uaad ia ouaerical analpala. Verp 
genarallp, a ralaaatioa pracaaa ia organiaad 
aa follower 

(a) A llat of pooaibla labala ia iniepen- 
dantlp oalactod for aach part, baaad 
OB ita iatrinaic character iotica. A 

neaaure of caofldaoce can aloo be 
aaaoclatad with aach poaalble label. 

(b) The poaaibilitiea (and coafidencee) 
for aach part are conparad with thoee 
for related parta, beaad on a nodrl 
for Che ralationahipo between the 
poaaible lebala of picture parte. 
Labala ere deleted or nodlfled or con- 
fldancao are adjuotad, to reduce In- 
conaiatanciea. 

(c) Step (b) can he itaretad ae aanp tlnee 
ae required. 

Tble approach ia varp gmaral: be Iwve not 
apaciiiad how to faioolata label ralatlonahlp 
•odala, cbooae poaalbilltiaa, att inote can- 
fideacea, or adjwat thm; wr twee we die* 
cuaaad when Che pracaaa ahould be Iterated, 
and if ao, hew mmny tinea. 

Thaae laauea ore diacuaaad ia acne deuil 

la |1), and, ia general, are a runctloa of 
the problw at hand. Far a general diacua* 
alen, the reader ahould coaaulc |t|. be 
will. Inataad. conalder two apaclflc prob)ene 
la detail - pUel claoall Icat ion and notion 
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41«p«lty HilaaltM. Tk* lliat Invelvaa • 
•rakalllc Ukal Mt mti lha aacead a aiaarlc 
aaa. Tka ftral laaalvaa taaf Uaaca a^jvataaat 
aa4 tha aacaa4, sMItlaaall;, UvaUaa m 41- 
llcallaa a( aaaarU lakaXa. (actUaa 1,1, 
aai 4 alll caaaUar alapa a,k, aa4 c aka«a 
(ar Ikaaa taa praklaa. 


II ana waia Iarca4 ta kacUa aa a ( Laa4 
lakal far a ptaal, i, tkaa aaa Maall chaaaa 
tka a far whuk p (a) la aaa laal . Aa aaa- 
llaaal aarllar, auck aa talapaalaat claaal* 
ficatlaa caatalaa aaap arrata. llpara la>t 
lllaairataa tkla. la tka aaat aactlaa aa 
4aactlka haw a aiapla raltaatlaa ptacaaa aaa 
tapraaa aa thaaa taaulta. 


2. Ikimi UtllL ASIlCmOlfT 

tka firal atap af a ralaaatiaa pracaaa la- 
aalvaa aaalpal^ Ultlal lakala ca aach plc- 
tara part. If tka pivaa lakal aat la aya- 
kallu, tkaa tkla la atllaarllp 4aaa ualap 
taclal^t fraa atatlatlcal pattata racagal- 
t>aa. Mara, aaaaitMaata ara caapatai far 
aack piciara part aal. kaaaf aa a arlari 
aaMala af tka claaa caaMItlaaal Maaaltlaa af 
tkaaa aaaairaaaata a prakakllttp tkat aack 
lakal la tka carract aaa far a pUtnta part 
caa ka caaputaM. Tkaaa prakakllltlaa aarva 
aa tka aaaauraa af caaflMaaca rafarral ta la 
tactlaa 1. taaa aaaaplaa af apakalllc lakal 
aata, L, aal aaaaclataM pictura part aaa- 
•vrsaata. N, ara: 

(1) la a "aprlag-laaMad* taiplata aatcklap 
praklaa |2] L MoaU ka tka aat af aak- 
taplata aaaaa aa4 M tka craaa-«arra- 
latlaa af taa auktMpUta at a parti- 
cular laaaa paaltlaa. laa |1| far 
MauUa. 


Ik aany apyllcatlaaa, tka aatural lakal aat 
la a auaatU prapartp ualua ratkar tkaa 
apakalllc, au4 tka lakal aat at aack pictura 
part la aftaa laltlallp rapraaaataM kp tka 
aaat curraat aatlaata af tka aaat llkalp 
prapartp valua at tkat pictura part aaf a 
aaaauta af caaflMaaca aftaa ralataM ta tka 
uarlaaca af lacal prapartp ualaaa. Par 
aaaapla, la ralaaatiaa alparlttea far trap 
laual laapa aakaacMaat, tka lattlal prapartp 
valua at aack ptaal la tka plaal'a pPap laral, 
aai tka aaaaura af caafMaaca, far placawlaa 
caaataat laaiaa, uuuM ka tka prap laval 
varlaaca In soar aalpkkarkaal af tha pictura. 


A aura caapllcataM aiaapla 1a aailaa 41a- 
parltp aatlaatlaa. Mara, tka laltlal prapartp 
value la a aat laa vactar uhlck caa ka c ca- 
put ad aa fallaua. At aack plaal, aaa caa 
ccaputa a llaaat caaatralat aa tka a aa4 p 
coapeaaata (u,v) af tkat plaal'a an clou kaaa4 
aa tka apuatlaat 


-*t * *.• * s* 


( 1 ) 


(2) t« 4ot clustct L • llnttflor 

«4|« polat, pelAt) N 

Ar« MAMrMMta o2 loc^l 4et 
Sm |4| for 

O) iB MperviMd yistl clABtlf icaileo, L 
ie tb« BBt of glvoa c1«m ombb, aii 
N orltoBrlly coot«l 0 B ipoccral oodp 
local tacvrol foocuroo. 

fwrtulat (lit ylstl cltotif lett loo In aort 
lollp FIfurt Itf obooB ols teoito of o UlClE 
loot Bit# (1473). ThtBt Bis iMgoB Brt froa 

thtM ClM BCgUlBltlOOB BSi tOD boodt (3 BOd 

3). Croiia4 trutb 1 b BtoUtbl# for cMb bUBp 
mmd it CBB b« wBotf to cosputa thm cIobb coo* 
4itloool prehobillty dooBitloo ot ooch tlso 
lor tlw *- 000 ( 0 ? of ooBOuf ooto ot ooch tiso. 

•y oo4ollog tWoo 4BsaitloB oo oovsol, m cob 
o4opc tlw followti^ Blaplo procorforo for 4o- 
tfOttlBlOlBg ClOBO probobllitioB fot oocb 
plstl at oBcb tteoi 


■oro, 1 1b tbo tMporol iBOgo IntooBlty 
chBBgt BBd 1 ood 1 roproooot tHo opttlol 
looft intODolty grolloota Vo ooouso Isro 
tbot gray lovol lo oo iovorloot of sotloo. 
AoBiaiiig that tbo laogo sotioo io locally a 
troBBlBtloBp thoo eot could ceobiot tbo o^- 
tiooB at two adJocoDt polsta to obtolo o 
uBlgut ooloclty ooctor ot ooch yolat, or» ooro 
gOBorally, ono coo eosputo tbo "paoudo-lottr- 
Boctioo** of tbo llnoor cesatroiotB lo o 
Btlghborhood of ooch piaol |S). lo tbla coot, 
tho error of tbo fit coo bo uoad oo o oooBuro 
of e oaf Id toco at ooeb piool, oltbougb oo will 
Igoort the cenfidoocoB lo wbat folloua. 

JLa on OBBoplOp Flguro 3 cootoloa too froota 
iB B l1o« vorylaf loogo, ood Flguro 4 con* 
toiOB thB laltlol velocity voctet cooputod 
by tbo pBoudo-latotooctloo tocbolguo. 


3a UUUUTICW 


(1) Cooputt tbo folloulat dlotooco mo- 
ourt of a plBBl. Bp froo o cIbbb, dt 

dj^(a) • loglt^l ♦ (i-y )* Ij^^(a-y ) 

boro, iB tbt oooa vector lor clovo 1 ood 
to Ita cevarloocB oatrla. 

(2) Mat, cosputt tbo probobLlUy that 
plsoi a btloofB to cloBB 1 by: 

Pj^(i) - U/dj^(«))fr|l/dj^.(«)l 


Altar iBltlol lobolllogB (ood coofldaocoo) 
art cooputod, ono boglea o bb^boco of Itoro- 
tlooB wboro tbo lobola ood coofldoocBo ot 
ooeb picture port aro oodUlod booed oo tbo 
dlBtrlbuttoo of lobola ood coofldoocoB oo 
tolatod picture porta ood apse oodol of bow 
labolllAgB effect ooo ooetbn. Thua, tbo 
rolOBotion olgorllls 1b booed oo: 

o) o oodol for the BolgbborhDod of o 
picture port, ood 
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k) * M<tl fat tiM latarKlloai ktiwaw 
kalllata af a<]aeaa( plctura parta. 

Tha aal|tikarkaa4 aadal far a ralaaatlaa a**- 
caaa apn **;.a «klch palta af ptctara parta 
dtractlp caaaalcala aitk aaa aaalhar la iPa 
ralaaatlaa practia, aa4 4aianlaaa tka tapa- 
lapp af tka prapk aa wkuk tkr ralaaatlaa 
pracaat aparataa. Tkla prapk kaa 1 b4I*I4m 1 
ptctara pacta aa aa4aa. lu area caaa act 
tkaaa paira af parta tkat ciaaiialtita alck 
aaa aaatkar. Tka aalpkkarkaai aa4al la aaa- 
allp 4aaljaa4 ta aatakliak caaaactloaa aalp 
kataaaa 'aaarkp' parta* 

* atlikkorkual ao4al la apacUlai kp a aat af 
aaipkkar eaUtlaaa r • (r,,r.,....r,). lack 
r. la a klaarp ralatlaa 4aflM avar ika ap- 
praprlata aat af ptctara parta. Par aaaapla. 
If tka ptctara parta ara plaala, tkaa tka 
aal«hkorkaa4 aa4al at|kt apaclfp tkat a plaal 
la caaaactal ta a* arp plaal Ita !■] aalpKkar- 
kea4. la tkla 'aaa, tkara ara atlll aavaral 
paaalkllltlaa far tka ralatlaaa caatalaa4 la 
Ika aat r. Par aaaapla, r Blpkt ka tha aat 
Ulractlp akaaa, 4lractlp kalaa, ate.) wkick 
aoiiU 41atta|vlak kataaaa paira af pataca 
that ara karlaaatallp a4]acaat, aartlcallp 
a4jacaat, au., ar It c«il4 ka tka alailataa 
talatlea *la tka |al aal|kkackea4*. la tka 
latcar caaa, tka caaaactlaaa kataaaa paira 
af plaala aaal4 act ka racavarakla fraa tka 
irapk aa akiek tka ralaaatlaa pracaaa all! 
aparaia. Tha ckalca af r will, la icnaral, 
ka lataralaaf kp tka laetrapp af tha aatvaraa 
af lakala. Par acaapla. If aa ara 4aat|alai 
a ralaaatlaa pracaaa far a4|a ralafercMaat, 
tkaa tha ralaclaa paaltlaaa af plaala ara 
crucial aiBca a4gaa paacrallp 'llaa up', 
aklla If aa ara 4aalgalag a ralaaatlaa pro* 
caaa ta aakaaca aa laapa'a trap lavala. tkaa 
tha poaltloaal lafaraaclaa aap aoc ka ra- 
tutraC. 

Whan tha plctura parta ara ragleat rather 
than plaala, than coaaactlaaa al|ht ha foraa4 
katuaaa aCJacaat raglaaa aalp. la aoia alta- 
allaaa. It aliht ka aaeaaaarp to 41atta|ulah 
katuaaa raglaaa tkat ara ahoaa. kclou, lnal4a, 
aurrouaClag, ate. 

Tha lataractlaa aa4al 4af|aaa hew a plctura 

pan chaagaa Ita label >g kaaa4 on tha 

lakallli^a af Ita nalghkara. la lataractlaa 
ao4al la caponai af twa parta; 

(1) a laawlagia raaraaantatlen far tha 
ralaclanahipa katuaan lakala, aad 

(1) a archaalJB, ar praca4ura, far applp- 
lat Ika teawlagga la (1) to chaaga, 
ar up4aca, laballlaga. 

Per 4lacraca apaholllc laballlaga tha ala- 
plaai knawlalga rapraaontaclaa la a aat af 
tka paira of lakala that can alaultaaaoualp 
ka aaaaclacag with paira af aalghbarlag plc- 
tura parta. It caa ka rapraaantal kp a 
klaarp ralatlaa g 4aflaa4 waar tka ualraraa 
af lakala D. Inculclaalp, (4,4') i ■ 


If a pair af na4kkara caa alaultaaaoualp 
ka laballa4 with 4 aa4 fi'. la gaaaral, tkara 
la a klaarp ralatlaa aaaaclaia4 with Mch 
aalghhar ralatlaa. 

Tha aaat abviaua up4atlag aachaala far 41a- 
crata, apaholllc lakalllag la a lakal 41a- 
car4lag pracaaa, whick laaka at paira af plc- 
tara parta at a tlaa. 4 USal, 4, caa ka 
4alata4 fraa tha lakalllag of a plctura part 
af, (ar aaaa aalghbarlag plctura part, that 
aalghhar 4oaa aat caatala a label, 4', la Ita 
UbaUlag with (4.4*) 1 B. 

Tha klaarp ralatlaa haawla4ga rapraaaatatlaa 
caa ka gaaarallaa4 to apaholllc laballl^a 
with caafl4aacaa fa* aach apaholllc lakal 

kp apaclfplM a raal^laa4 r aa p atlkUltp 
(uBctlaa, C, whaaa 4aoala la B4. 4a bafara, 
ta gaiaral, a cspatlkllltp fuactlao la 4a- 
(laa4 (ar aach ptctara ralatlaa la tha aat r. 
4 warlatp af appllcatlaoa haua uaa4 caapatl- 
kllltp fuactloaa whaaa raaga la (-1,1). la- 
lultWalp, U C(4,4') - -1. tte 4 a^ 4* ara 
aaalaallp Incoapaitkla, aa4 tha acraag pra- 
aaaca af 4* at aaa plctura part (l.a., 4* 
kaa a high llkallhaaC at that part) ahaul4 
4apraaa tha llkallhna4 af 4 at a aalghkorl^ 
plctura part. If C(4,4') - 1, rhaa 4 aa4 4' 
ara Baalaallp caapaclkla, aa4 tka atraag pra- 
aaaca af 4* at a plctura part ahaulB iacraaaa 
the llkallhao4 of 4 at a aalgkharli^ plctura 
part, riaallp, 11 C(4,4') - 0, than tha pro- 
naaea a! 4' at a plctura part aheul4 hava no 
affaet aa tlia llk^lhoel af 4 at a aalghhor- 
lag part. Iataiaa4laca valuaa of C ahoul4 
hawa lBCaraa41ata alfacta. 

Sauaral aackaolHa hawa kaca auggaatad far 
applying tkla knowlodga rapraaaatatlaa to 
upCatlag lakallinga. far aaaapla, gaaaafalg 

at al. [1] auggaatad tha formula 

p;(4) • p^(4)(l * q,(4))/a (1) 

wkaro 

Q,(4) - r a„ I C(4,4*)p,(4') O) 

I j IJ J 

aad W la a aoraallalag (actor which guaraaiaaa 
that p (4)-l. Tha a . valuaa caa ka uaad ta 
give klghai weight ca'aaaa aalghbora at part 
I tkaa atlraca. Q.(4) aaaaaraa tka ovarall 
auppert af tha nalgkkarhaod af part I (ar 
lakal 4; It takaa aa valuaa la tha raaga 
1-1,1) aad caa ka Intarpratal alallarlp to C, 
Tka ahiva oparatlaa la applied la parallal 
at ovary part aad (or avery lakal. Tka p' 
valuaa than raplaca tha p valuaa, aad tha 
oparatlaa caa ka llaralad. 

Vary afcaa, kouavar, a gaaaral updatlag rule 
Ilka (2) la Inappreprlata kacauaa It (alia la 
taka advantage af knewladga akaut tha apacl- 
flc proklaa at kand. far aaaapla, la tha 
crap claaalf Icatlaa praklaa aavaral plauolhla 
rolaaatloa rulea can ka derived fraa haurlatic 
argtaaBta haaad aa tka fallowing gaaaral ak- 
aarvatlaa: 
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ritlii an. In inwint, bikIi kl 4 |«r ttv* • 

• fiul nB4 *0 cknat* clann 4urla( 
a alaala iiawtat aaaasa. 

II m lat 4aaaia Ika ^rakakllliy 

that Ika riaal al a^aM*> Ucallaa (l,J) aa< 
tiM I la la claaa I aftat k llaratlaaa af 
Ika ralaaailaa nla, Ika tka fallawlat r«la 
ackaa can ka aaai ta 4arlaa aarlaaa apacl- 
llc nlaai 

• K(»J.<l\J'.i)i k'U. (I'.J*) • 
aal|kka> af (I.JlS' 

W1 

■aaa, II la a aaraallaallaa facta; <• I 

(I•J•<)) •*<< * la tka aat af faaalkla claiaaa. 
Tka aacanf tata rallaria Ika fact tkat tka 

claat al a plaal 4aaa aat ckaa(a avar tkna, 
lAUa Ika tktrf lara ■aat cartaaaai tka afa> 
llal fayaafaara af (l.J) kal^ la claaa 1 
aa tka lUallkaal tkat lia aalakkara ata la 
aailava claaaaa. Ha can IHaatlfj al laaat 
tteaa alaaalkla fcactlaaa lac fi 

1) f. • Baa K(a.,t). 

* 1-0 ‘ 

Hate, a, ta aa t-aal|kkat al (],k) (aaa 
fiaara kalaa) aaf Ikla tala caa ka latatytaial 
aa aa 7 la|! It (J.k) la la claaa 1. Ikaa atacc 
flalla arc lai|a, at laaat aaa a^iul aalak- 
kai af (],k) Bial ka la claaa 1. 
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•» 


•l 

•l 

*1 

•l 


' ‘ k 

1) I, - aaa D P>(C(«,>t)t aakaetlkta ao4 0. 
* 1-0 1-0 * ' 

If flaUa ata raagkly lactaafulac, tkaa aai 
fatal all! fcaaa at laaat tkraa caaaacativa 
aalfkkata la tka ttmt flaU. Matica tkat 
alafly ckaaalac 

aauU at fliat Itaiallaa laaat Ika frakakllt- 
llat af cerract lakala at kartcc flaala, aaO 
ptapatata ikaaa laa piakakiltlaa lata tka 
caalar al tk« flaU at aakaac — t Icatatlaaa. 

».I k 

" '> • ,io 

Oaa polaatial aCwantaga af tka aua aaar tka 

praCact la Ita laaaiaUlvltf ta aaa at cw 
arroaaawalp laa prakakllll laa. Ualap f^ alaa 


plvaa aaia ataal aal^kt la Ika laparal anf 
apallal lalaraatiaa la tha ralaaailaa rale, 
la Ika aaal aactlaa aa atll caaalOat Ika 
aflatt af applyiaf tfcia apOallaf tala ta 
Ika laaiaa la ri«ara 1. 

Heat. taaaUar tka bbiIm aatlaailaa prakla. 
Al aack plial aa kavc «V tka aatlaaia af 
Ika vclaclip after tka k<k iiarallaa af 
ralaaatlaa. Naa, aa alll aaawa tkat. 
lacallp, Ika paltora at laaga aatlaa vactara 
caa ka aall iaacrlkaO kp a rlfU laaia plaaa 
aatlaa caaalatlai af a traaalatlaa aal rata* 
tlaa (aaar lar|c raiiaaa tkla aifkc ka a 
pact aaaaapilaa alaca tka laapa aatlaa la 
Ika pialacliaa af a 1-0 rIaU aatlaa). 

GaaaMac a I*! aalakkarkaaO af valaclllca 



akara la tka valacltp at p^. 

Ilaca aay 1-0 rltU aatlaa caa ka rapraaaaiai 
aa a traaalatlaa plaa a ratatlaa akaat a 

flaa< palal. If aar aaaaptlaa at lacal aa- 

ilaaa kalag rlalA laapa plaaa aatlaaa la 

carract, ilw tka 1>0 paitara af aalacitlaa 
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'•"*( 

0 

**"•( 

k-( 

*»■•( 
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akaulf ka a ratatlaa akaat tka caalar petal, 
la •aacral. If potai a ta retatlac akaat 
polai k alik aaloctip v^, tkaa 

a • 4. • 0 

a ka 

akaa 4p, la tka vactar fraa k ta a. Tkara- 
lara. li la atialtktfaraarl ta caapvia a 
iMat acuaraa aatiMla af tka aapalar aalacl- 
tp, a., el pg.la aap JnJ (ar a^) laata 
aatpkaarkaaf . 

U t9 C«pyt« • «• 

pf€99i wUb th* ^rallU •^r«- 

tlMt Ueb lb • mm%9r •! cIm Mt«t 

flat I SO oaltkbQrbM^* (aat tmtvmi 

•( Mcb of Ito i ooltliboro). CIaom tiM 
Mlflibor , 1. vlU ■talaol loooC ^rrt 

oob oot ♦ *j. 

4 . XTUATIOH 

Ao laporconi probloo &■ opplicotloa of 

roloaot ion procaaooo io bocoratoinft • tot* 
olootioo crliorlo lot tbo ttorotlooo. Tbo 
two obvioMO tomlaotloo crltorio oio: 
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I) "(MVatlMC** IB Ik* t«M thti tkt 
ckMit (U rr*takllU7 Mf 

MrU >*k*X«, nc.) Iia Uaraliaa 

!• Ik« ■Mt it taffUitMlf wll 
(tta |t| IM t tIaciitalM •( caavar* 
■*at a talatallaa praaataat). 

» aakaatilat tUacalai caaytatla a a l 
tattHfcaa - a.|., la aattaa tetaatlaa, 
aat aaf aalj kava a tkart tlat (< 1 
r t) la llaratt tka ralaaatlaa pracaaa. 

(aattlaaa aaa llait tkat afellt tka liaat ta> 
varal Uarttlaat at a ralaaatlaa ^racata 
'^laa ta l^tart ufm tkt lalilal lakallai, 
lilavlai tka Ittratlaaa la pracaai actaally 
4a|raaat tka raaalta. la aack caaaa aaa 
aaaU «aat ta ttralaatt altar aaly a faa 
Itatatltaa. 

rat ika Mltl-tafaral, Biltl'tpactrtl pUal 
cUatiftcatlaa, at Mai Malrlctll)p. tktl 
tka laitaai lactaaaa la claaalf Icatlaa ac* 
caiacr accara la tka llttl Itarttlac. aitk 
aakatiaaai itarttlaaa karlag llttla affaci 
aa tka raaalta. ri(ara % caatalaa tka 
clatall Itti laa aaaa far tack cajar data la 
ri|art 1 altar i Itaratlaat al tkt raiaaa- 
tlaa aliarltte iaacilkat la Sactlaa 1 aalag 
laactlaa I}. rraa tkata llaaraa. It la 
claar tktl tka ralaaatlaa al«atltte kr. aak- 
atattlallr aakaacai Ika iaitdlta a< tki 
llalia aai tka araiatl claaallleatlaa ttia- 

■aai, caaaliar tkt aailaa tatlaatlaa rtak- 
laa. kart, tkt crlllcal Itctar la iatar- 
alalaa Ika aaakat al Itatttlaaa ol tka ra- 
laiatlot kractta la tiaa, alaca Irtata ara 
arrialai aaar> .01 aacaaia. rigara 1 alma 
Ika aailaa raciart altar 1 Itaratlaaa al 
ralaiallat. *llkea|k tkt ractata la ri(art 
t arra*' aaalkar Ikr.a tkata al ri|ara 1, 
aaa aaaia a«a taaalttatlaa aaaaara lar 
CMaarlat ikM. Caa awk aaaaara la ta aaa 
kaa aall tka aalacltr aaciara arailci laaga 
airactara (tar, lar caiai traaaalaalaa) . 

II t(I.t),l-l,I la ka iMtaaltr al ala*l 
V at llaa t, tkaa l.i 

a^(W) - ||(a.l)-i<aaa^l)| 

aai, llaaUr, 

1^. la^d) 

II tka ralaaailaa la tralr laaraalat tka 
ralacIlT aaciara, Ikaa aa aaali aaaact 1^ 
la ka BaaataaUallir aaalacraaala( with k. 

rifan I akaaa 1. lar k»0 1 lac avr 

aaa^l.'. 


COMCIJimMC IBMIU 

■alaaatlaa aracaaaaa haaa raiaatlal apaai 
aiatataiaa kacaaaa Ikap caa ka laplaaantai 

la parallal (kaciaara paraicilaa). Tkar 

kaaa kaaa aaccaatlallp appllai ta a alia 
aarlair al lakalllaa praklM kp a iraalaa 


■Makar al laraatliatara. la tplit al 
tkata taccaaaaa, llttla la at pat kaaaa 
akaat tka iatl(a aai caairal al tkata pra- 
caaaaa. kaaarar, a aaa k a r al praaltlaf ap- 
praackat ta ikair Ikaarailctl laraalailaa 
ara kalap paraaai, aai It la kapai tktl a 
iaapai vaitrataailat al Ikalt aalara irtll 
aaaa ka teklarai. 


fkt aappart al tkt katlaaal tclaaca Paaaia* 
tlaa aaiar eraai.n-Tt-IMtl U iraialaUp 
aclaw*lai|ti, aa la tkt kalp al Jaaat lalaa 
la prcparlap tkla papar. 


1. DarU, 1. t.. aai i. laaaBlali (IHl). 
Caaprrat pracaaaat lar la»-laral 

rlalaai i aarrap. artlllclal latalll - 
aaact . 17, Ii»-lU. 

a. Pltcklar, M., aai t. Ilaeklapar (1*71). 
Tka rapraaaatatlaa aai aatcklap at plc- 
tarlal laactlaat. IT U C aaaatara. 11 . 
»7>P2. 

1. Ikalt, L. I., a^ A. laaaalaM (1171). 
ApplUatlaaa al ralaaatlaa lakallap: 
Iprlaa-laaiai taaplata aatcklap. It 

Prac. Iri let. Jalat Oaal. aa Pattart 
kacapaittaa, laa Dlapa, CA, lll>kl7, 

A. lacker, I. W., aai I. A. laMl (1171). 
Taaari a laa-laaal itacrlptlaa of iat 
claatarat lakcllap aipa, latarlar aai 
aelac polaia. Caat M tar Craaklca aai 
laaaa Ptacaaalaa . >, 113-1)1. 

1. Llak, J.. aai J. Mirpkp (1I7S). btl- 
aatlap tka ralacltp al aarlap laapaa 
la TV alpaala. Procaaalaa . i, 111-117. 

A. keamlali. A., t. A. luBcl, aai 

I. V. lacker (1I7A). Icaaa lakallap 
kp raltaailaa aparatlaaa. HUT-iWC . 

A, 11»-«11. 


Plpara 1. Tkraa acpalaltlaaa al 
lar 1 kaaia 


I 


ORIGINAL PAGE fS 
OF POOR QUALITY 




fl|urt 5. Labeling after 5tb iteration 
ueing i.« 


figure 2. Initial labeling (left celuan 
ie ground truth, right containe 
glaele elaealfied into that 
ground truth elaaa) 


fifurt 1. Two franea of a traffic aequence 


figure 6 


Initial notion voctora 


figure 7. Normliaed valwea of 
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Dual Problems In Image Segmentation 


Jack Bryant* and Susan Jenson 


Summary . The obvious duality between edge finding and segmentation 


was exploited in [1]. This work has been refined, including a more 


general model for multi-image data, and many more tests of the cluster- 


ing program AMOEBA. Possible directions for future work in edge follow- 


ing and clustering are suggested, using (now) the duality between seg- 


mentation and classification made possible by regarding AMOEBA as a 


segmentation to c) asslflcatlon mapping. 


1. J. Bryant, On the clustering of multidimensional pictorial data. 
Pattern Recognition 11 (1979), 115-125. 


* Jack Bryant, Department of Mathematics, Texas A&M University, College 
Station, Texas 77843. 


+ Susan Jenson, Applications Branch, EROS Data Center, Sioux Falls, South 
Dakota 57198. 
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SHAPE AND TEXTURE 
Azriel Rosenfeld 

Computer Vision Laboratory 
Computer Science Center 
University of Maryland 
College Park, MO 20742 

Regions in a segmented image are characterized, for purposes 
of description and recognition, by their geometrical properties 
(size, shape, etc.)f as well as by properties that depend on their 
pixel values (lightness, color/spectral signature, texture). Such 
properties are also used to define or modify the segmentation pro- 
cess itself, as discussed in the session on segmentation. 

The methods used to measure the geometrical properties of 
a region depend on the data structure used to represent the region. 
The simplest representation is a binary "overlay" array that has I's 
at region pixels and O's elsewhere. However, other types of repre- 
sentations are often used that are more compact, and that may make 
it easier to extract certain types of geometrical information. One 
classical approach is to represent regions by border codes, defin- 
ing the sequence of moves frexn neighbor to neighbor that must be 
made in order to circumnavigate the border; curves can also be 
represented by such move sequences ("chain codes") . Another stan- 
dard way of representing regions is as unions of maximal "blocks" 
contained in them - e.g., maximal "runs" of region points on each 
row of the image, or maximal upright squares contained in the re- 
gion; the set of run lengths on each row, or the set of centers 



/ 
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and radii of the squares (known as the "medial axis"), completely 
determines the region. The square centers tend to lie on a 
set of arcs or curves that constitute the "skeleton" of the 
region; if we specify each such arc by a chain code, and also 
specify a radius function along the arc, we have a reprasenta- 
tion of the region as a union of "generalized ribbons", which 
are 2D analogs of the "generalized cylinders (or cones)" often 
used to represent 3D objects. 

There has been recent interest in the use of hierarchically 
structured representations that incorporate both coarse and 
fine information about a region or feature. A hierarchical 
maximal-block representation based on recursive subdivision into 
quadrants, where the blocks can be represented by the nodes 
of a degree-4 tree (a "quadtree") , is described in (S 2 unet and 
Rosenfeld, 1980) . A hierarchical border or curve representation 
based on recursive polygonal approximation, with the segments 
represented by the nodes of a "strip tree", is discussed in 
(Ballard, 1981); on a border or curve representation based on 
quadrant subdivision see (Shneier, 1981) . 

Classically, textural properties have been derived from 
the autocorrelation or Fourier power spectrum; for example, 
the coarser the texture (in a given direction) , the slower its 
autocorrelation falls off in that direction from the origin 
(zero displacement) and the faster its power spectrum falls 
off in that direction from zero frequency. A related approach, 
studied extensively by Julesz and Haralick, characterizes tex- 
tures by their second-order intensity statistics, i.e., by the 


frequencies with which given pairs of gray levels occur at 
given relative displacements. It has long been realized, 
however, that first order statistics of various local proper- 
ty values (e.g., responses of operators sensitive to local 
features such as edges, lines, line ends, etc.) are at least 
equally effective in texture discrimination. 

More recent work (Beck et al., 19b2) suggests that local 
processes of linking between local features, giving rise to 
"texture elements" or "primitives", also play a significant 
role in the perception of texture differences. Texture dis- 
crimination based on second-order statistics of local features 
(e.g., occurrences of edge elements in given relative posi- 
tions and orientarions) has begun to be investigated (e.g. , 
Davis et al. • 1979) . Texture analysis based on explicit ex- 
traction of primitives has also been explored (e.g., Maleson 
et al. , 1977); here statistics derived from properties of 
the primitives, or of pairs of adjacent primitives, are used 
as textural properties. 
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SHAPE AND TEXTURE 
Azriel Rosenfeld 

Computer Vision Laboratory 
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t 

Regions in < segmented image are characterized, for purposes 
of description and recognition, by their geometrical properties 
(size, shape, etc.)f as well as by properties that depend on their 
pixel values (lightness, color/spectral signature, texture). Such 
properties are also used to define or modify the segmentation pro- 
cess itself, as discussed in the session on segmentation. 

The methods used to measure the gecmetrical properties of 
a region depend on the data structure used to represent the region. 
The simplest representation is a binary "overlay" array that has I's 
at region pixels and O's elsewhere. However, other types of repre- 
sentations are often used that are more compact, and that may make 
it easier to extract certain types of geometrical information. One 
classical approach is to represent regions by border codes, defin- 
ing the sequence of moves from neighbor to neighbor that must be 
made in order to circumnavigate the border; curves can also be 
represented by such move sequences ("chain codes") . Another stan- 
dard way of representing regions is as unions of maximal "blocks" 
contained in them - e.g. , maximal "runs" of region points on each 
row of the image, or maximal upright squares contained in the re- 
gion; the set of run lengths on each row, or the set of centers 
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and rad..i of the squares (known as the "medial axis"), completely 
determines the region. The square centers tend to lie on a 
set of arcs or curves that constitute the "skeleton" of the 
region; if we specify each such arc by a chain code, and also 
specify a radius function along the arc, we have a representa- 
tion of the region as a union of "generalized ribbons", which 
are 2D analogs of the "generalized cylinders (or cones) " often 
used to represent 3D objects. 

There has been recent interest in the use of hierarchically 
structured representations that incorporate both coarse and 
fine information alaout a region or feature. A hierarchical 
maximal-block representation based on recursive subdivision into 
quadrants, where the blocks can be represented by the nodes 
of a degree-4 tree (a "quadtree") , is described in (Samet and 
Rosenfeld, 1980) . A hierarchical border or curve representation 
based on recursive polygonal approximation, with the segments 
represented by the nodes of a "strip tree", is discussed in 
(Ballard, 1981) ; on a border or curve representation based on 
quadrant subdivision see (Shneier, 1981). 

Classically, textural properties have been derived from 
the autocorrelation or Fourier power spectrum; for example, 
the coarser the texture (in a given direction) , the slower its 
autocorrelation falls off in that direction from the origin 
(zero displacement) and the faster its power spectrum falls 
off in that airection from zero frequency. A related approach, 
studied extensively by Julesz and Haralick, characterizes tex- 
tures by their second-order intensity statistics, i.e., by the 





frequencies with which given pairs of gray levels occur at 
given relative displacements. It has long been realized, 
however, that first order statistics of various local proper- 
ty values (e.g., responses of operators sensitive to local 
features such as edges, lines, line ends, etc.) are at least 
equally effective in texture discrimination. 

More recent work (Beck et al., 1982) suggests that local 


processes of linking between local features, giving rise to 
"texture elements" or "primitives", also play a significant 


crimination based on second-order statistics of local features 


(e.g., occurrences of edge elements in given relative posi 


1979) 


Maleson 


et al., 1977); here statistics derived from properties of 
the primitives, or of pairs of adjacent primitives, are used 
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SHAPE AND TEXTURE 

Azriel Rosenfeld 

Computer Vision Laboratory 
Computer Science Center 
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a) Moments 

In this section, we review moments, which are 

a very useful class of shape properties. 

The (i,j) moment of f is defined by 

m.. = E Zx^y^f(x,y) 

^ X y 

(in the continuous case, El becomes //dxdy). The first few 

moments of the picture 

2 11 
3 10 
3 2 1 

are as follows, if we take the origin at the pixel in the 
lower left-hand corner of the picture: 


Moments can be given a physical interpretation by regarding 

gray level as mass, i.e., regarding f as composed of a set 

of point masses located at the points (x,y) . Thus m^^ is the 

total mass of f, and mp 2 ™20 moments of inertia 

of f around the x and y axes, respectively. The moment of 

2 2 

inertia of f around the origin m^ = EE(x +y ) f (x ,y ) =m 2 Q+mQ 2 • 

It is easily verified that m is invariant under rotation of f 

o 
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about the origin. 


Moreover, if f is 


rescaled, say by the factor c, it is not hard to see that 
is multiplied by . Thus we can normali zc f with respect 
to magnification by rescaling it to give m^ a specified value. 

Alternatively, a ratio of two moments 
that have the same value of i+j, e.g. ”'oi^"‘lO' invariant 


under magnification. 


If we substitute -x for x in the definition of m^^ , we 
obtain ZZ (-x) ^y^f (-x,y) = (-1) ^Hx“y^f (-x, y\ so that if f is 
symmetric about the y axis (i.e., f(-x,y) « f(x,y) for all 
x,y) , we have m^^ = (-l)^m^j. Thus if i is odd, m^^ must be 
zero. Similarly, if f is symmetric about the x axis and j is: 
odd, m^j=0; and if f is symmetric atbout the origin (f(-x,-y) = 
f(x,y) for all x,y) , and i+j is odd, m^^ = 0. Moments for 
which i, j, or i+j is odd can thus be used as measures of 
asymmetry about the y axis, x axis, and origin, respectively. 


If f is a binary-valued picture, say with S as its set of 
I's, the moments of f provide usefu] information about the 


Li 


1 
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spatial arrangement of tne points of S. To compute moments 

from the binary array representation X3 of S, we simply sum 

the x^y^ values for all (x,y) in S. To compute them from 

the run length representation of S, we compute them for each 

run and sum the results; for example, the (i,j) moment of 

the run whose endpoints are (x* ,y) and (x",y) is y^ Z x^. 

• x»x ' 

Similarly, they can be computed from the quadtree 
representation of S by computing them for each black leaf 
node, based on its position in the tree, and summing the 
results. They 

are not easy to compute from the MAT representation, since 
the blocks overlap. They can be computed from the crack or 
chain code representations of the borders of S in much the 
semte way that area is computed from these representations. 

As an example, for each horizontal 
crack Cj^, let Sj^ be the vertical rectangle of width 1 extend- 
ing from the bottom of the picture to Cj^; then S is the union 
of the Sj^'s for which Cj, is an upper boundary of S, minus the 
union of those for which Cj^ is a lower boundary. The coordi- 
nates of Cj^ determine the moments of Sj., just as in the case 
of runs; to compute the (i,j) moment of S, we add the (i,j) 
moments of all the upper-boundary Sj^'s, and subtract the sum of 
the (i. ,j) moments of all lower-boundary Sj^'s. 



I 
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b) The centroid; central moment! 

The centroid of f is the point (x,y) defined by 

* - "io''"oo 
y ' ”ol'''”oo 

Thus the centroid of the 3 by 3 picture shown earlier is 
(4/7, 6/7). It is easily verified that if f is shifted, its 
centroid shifts by the same amount. ( Proof ; If we shift f 
by (a, 6), the origin is now at (-a,-B), and the new coordi- 
nates of (x,y) are (x+a,y+B) . Hence El (x+a) f (x,y) /ZIf (x,y) ■ 

Q« x+a, and similarly for y . ) Thus if we take the origin 
at the centroid of f, we have normalized f. with respect to 
translation. Note that since the centroid does not have 
integer coordinates, if we take it at the origin we should 
redigitize f; alternatively, we can normalize f by taking the 
origin at the integer-coordinate point closest to the centroid. 
(Analogous remarks apply in the case of normalizing with 
respect to magnification in subsection (a).) 

When we take the origin at the centroid, moments computed 
with respect to this origin are called central moments , and 
will be denoted by m Evidently ™oo“*"oo' 

verified that ®io“”oi“°' Take (a , B) « (-x, -y ) in the 

preceding paragraph to obtain m^^Q = ZZ (x-x) f (x,y) « mj^p-xm^^ a 0, 


and similarly for ) 
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c) The principal axis 

The moment of inertia of f about the line 
(y-6)cos6 > (x-a)sin6 , which is the line through (a, 6) 
with slope 6, is 

££[(x-a)sine - (y-B) cos6l ^f (x,y) 

We can find the a, 6, and 6 for which this is a minimum by 
differentiating it with respect to a and 6 and equating the 
results to zero; this yields 

££[(x-a)sin6 - (y-B) cos0] f (x,y) = 0 (from 3/3a or 3/3B) 

££[(x-a)cos6 + (y-B) sin6] f (x,y) * 0 (from 3/36) 

Multiplying the first equation by sin6, the second by cos6 , 

and adding gives ££ (x-a) f (x,y) ■ 0, so that a*££xf (x,y) /I£f (x,y) 
Similarly, multiplying the first equation by cos6, the 
second by sin0, and subtracting gives 6=y. Thus the minimum- 
inertia line passes through the centroid of f. This line is 
called the principal axis of f. 

To find the slope of the principal axis, ta)ce the origin 
at the centroid; then the moment of inertia of f about the 
line y=xtan6 is 

2 2 — — 2 
££ (xsin 6 -ycos 0 ) f(x,y) ® m^pSin 6 - 2 mj^j^sin 6 cos 0 +mp 2 Cos 6 

Differentiating this with respect to 6 and equating to zero 

gives 

— — 2 2 — 

2 m 2 Qsin 0 cos 0 - 2 mj^j^ (cos 0-sin 0 ) - 2 mQ 2 COs 6 sin 6 * 0 

or m 2 Qsin 20 - 2roj^^cos26 - mQ 2 sin 26 = 0 



so that 


tan26 ■ 2m ,/ (m_ 


- m-,) 
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Since tan26 = 2tan0/ (l-tan^S) 


of the quadratic equation 

tan^e + tane 


m 


11 


, we can obtain tan6 as a root 
-1 » 0 


It is easily seen that the last equation above is equivalent to 

2 2 
(mj^j^tan6+m2Q) - ^™20^"‘o2^ (mj^j^tane+ni2Q) + ^®20"‘o2~"‘ll ^ ° 

This implies that m^j^tan0+n2Q is an eigenvalue of the matrix 
®20 “llj 

“ll “02 

Show that the principal axis is in the direction of the eigen- 
vector correponding to the larger eigenvalue of this matrix. 



A standard method of normalizing f with respect to rota- 
tion is to rotate it so that its principal axis has some 
standard orientation, say vertical. (Here again, this involves 
redigitization.) More generally, f can be normalized with 1 

respect to various types of geometrical distortions by trans- 
forming it so as to give standard values to various combinations 
of its moments. 

The principal axis of f can be regarded as a ' 

line that "best fits" f. More generally, one can find ^ 

higher-order curves that "best fit" f in various senses. For | 

\ example, given a general quadratic curve I 

2 2 

g(x,y) = ax + bxy + cy + ux + vy + w = 0 

L 

t I 

F 

t 

J 



6 


we can attempt to find the values of the coefficients a, b, 
c, u, V, w such that 

Zi:(q(x,y))^f (x,y) 

is a minimum. The curve having these coefficients would be 
a sort of "quadratic principal axis" for f. Given f's best- 
fitting quadratic curve q^, one can attempt to perform "shape 
normalization" on f by transforming coordinates so that qQ 
becomes some standard type of curve — for example, if q^ is an 
ellipse, we could transform to make it a circle. 
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.2. Texture 

This section discusses statistical picture properties, 
and in particular, properties that can be used to describe 
the "visual texture" of a picture, or better, of a statistically 
homogeneous region in a picture. He will not attempt to de- 
fine conditions under which a region would be called uniformly 
textured. Such regions are often described as consisting of 
large numbers of small uniform patches, or "primitive elements," 
arranged according to "placement rules," where the patch 
shapes and positions are governed by random variables. 


^ I ^ J 
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a) Gray level statistics 

The histogram p^(z) of a digital picture f tells us how 
often each gray level occurs in f; it provides an es limate 
of the gray level probability density in the ensemble of 
pictures of which f is a sample. If there are k possible 
gray levels, p^ is a k-element vector. Statistics 

computed from p^ give us general information about this gray 
level population. For example, 

1) The mean gray level of f, ^ Zzp^(z), where 

N = Z?^(z) is the number of points in f, is a measure of the 
overall lightness/darkness of f. The median gray level, i.e., 
the gray level m^ such that (about) half the points of f 
are lighter than m^ and half are darker, is another such 
measure . 

2 1 2 

2) The gray level variance of f, s p^{z), and 

the standard deviation o^, are measures of the overall con- 
trast of f*; if they are small, the gray levels of f are all 

dose to the mean, while if they are large, f has a large range 
of gray levels. Another such measure is the interquartile 
range r^, which is defined as follows: Let m^^ be the gray 
level such that 1/4 of the pixels of f are lighter than m^^^ 
and 3/4 are darker; let m^^ be defined analogously, with the 

*Note that if we define one-dimensional moments by m^ = ZzV^(z), 

we have N = m^ and that p^ is the centroid of Pj) ; 

moreover, if we define central moments m. = Z (z-p,) ip , (z) by 

2 ^ ^ ^ 

taking p^ as the origin, we have = m»/niQ 
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b) Second-order gray level statistics 

Statistics computed from the histogram are of only 
limited value in describing t, since remains the seune no 
matter how the points of f are permuted — for example, p^ 
is the same when f is half black and half white, when f is 
a checkerboard, or when f consists of salt-and-pepper noise. 
More insight into the nature of f is obtained by studying how 
often the possible pairs of gray levels occur in given rela- 
tive positions. 

Let ^ = (Ax, Ay) be a displacement, and let be the 
k-by-k matrix whose (i,j) element is the number of times that 
a point having gray level occurs in position ^ relative to 
a point having gray level z^, 1 ^ i,j s k. For example, if 
f is 

112 2 
0 2 2 1 
0 0 2 1 
10 0 1 

and ^ is (1,0), then is 

2 12 
111 
0 2 2 

Note that the size of depends only on the number of gray 
levels, not on the size of f. Elements near the main diagonal 
of Mg correspond to pairs of gray levels that are nearly 
equal, while elements far from the diagonal correspond to 
pairs that are very unequal. 
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Let be the number of point pairs in f in relative 
position this is less than the total number of points in 
f, since if (x,y) is near the border of f, (x+Ax, y-»-Ay} may 
lie outside f. Then in the matrix (i.e., if we 

divide each element of by N^) , the (i,j) element is an 
estimate of the joint probability that a pair of points in 
relative position £ will have the pair of gray levels ( 2 ^, 2 ^). 

Pg is called a gray level cooccurrence matrix for f. 

The matrices Pg, for various 6's, provide useful informa- 
tion about the spatial distribution of gray levels in f. For 
ex 2 unple, suppose that f is composed of patches of approximately 
constant gray level of a certain si 2 e s. If the length of 6 ^ 
is small relative to s, then the high-valued entries in P 
will be concentrated near its main diagonal, since a pair of 
points ^ apart will often have nearly the seune gray level. 

On the other hand, if ^ is long relative to s, the entries in 
P will be more spread out. If f consists of elongated streaks 
oriented in a given direction, the spread of values in P^ will 
depend on both the length and slope of If directionality 

is not important, we can use matrices P that are averages 

of ?g's (or matrices M that are sums of M^'s) for sets of 
displacements of a given si 2 e in various directions. For 
example, if f is the 4-by-4 picture shown above, and we use 
the displacements (1,0), (0,1), (-1,0), and (0,-1), then the 


/ 
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combined matrix M is 

8 4 4 
4 6 5 
4 5 8 

(Note that M is symmetric, since the set of directions used 
is symmetric.) 

In principle, a large set of matrices is needed to 
completely specify the second-order gray level statistics of 
f. In practice, however, matrices corresponding to large 


displacements are not necessary. As 6 becomes longi the 
pairs of gray levels separated by £ become uncorrelated, 
and Pg(i,j) approaches the probability that a pair of ran- 
domly chosen points of f have gray levels and z ^ . Thus 
for practical purposes we need not use £'s having lengths 
greater than the distance over which f's gray levels remain 
correlated, or greater than the size of the "patches" of 
which f is composed. In fact, the most important Pj's are 
usually those for which 6 has length 1. Historically, gray 
level transition probabilities p(Zj[z^) have been used to 
characterize textures; p(Zj|z^) is the probability that a 
point has level z^ given that the preceding point (with 
respect to a scan of the picture) has level z^^. Note that the 
joint probability p(z^,Zj), which is equal to p ( z^ ) p (z ^ | z^) , 
is just the (i,j) element of for ^ = (1,0). Other investi- 
gators have characterized textures by fitting a time series 
model to the sequence of gr~y levels, and using the parameters 
of this model as texture descriptors; this approach will not 
be discussed here in detail. 

Haralick has suggested a number of statistics 

that can be used to describe ^ given cooccurrence matrix P^ . 
Four of tliese are; 

1) "Contrast", I I (i-j ) ^Pg (i, j) ; this is the moment of 

i j 

inertia of P^ about its main diagonal. Evidently, 
it is low when the diagonal concentration of P^ is 
high, and vice versa. 
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2 

2) "Inverse difference moment", £ ZP. (i, j)/ (1+ (i-j) ]; 

1 j « 

this is high when the diagonal concentration is high. 

2 

3) "Angular second moment", Z ZP.(i,j); this is lowest 

i j ° 

when the Pg(i,j)'s are all equal, and high when they 
are very unequal, so that in particular it tends to 
be high when the diagonal concentration is high. 

4) "Entropy", -Z ZP . (i , j ) loi./I . (i » j ) ; this is highest 

i j 

when the Pg(i,j)'s are all equal, and hence is low 
when the diagonal concentration is high. 

It should be pointed out that the arrangements of values 
in the cooccurrence matrices depend not only on the coarseness 
or busyness of the given picture, but also on its lightness and 
contrast. For example, if we stretch the grayscale of a pic- 
ture, the entries in the matrices will spread away from the 
diagonal, since the pairs of gray levels will be farther apart. 
Features (1-2) defined above will be especially sensitive to 
such changes (this is why feature (1) is called "contrast"), 
while features (3-4) will be less so. To avoid confusing the 
effects of the first and second order statistics of the picture, 
it is -common practice to normalize its grayscale (e.g., by 
histogram flattening) before computing the 

matrices, so that the first order statistics have standard 
values. 

We can define cooccurrence matrices that may be more 
sensitive to the spatial structure of the given texture by 
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using only selected pairs of points in constructing the 
matrices, rather than using all possible pairs having a given 
relative position. For example, suppose that we consider 
only point pairs (Q,R) in which Q is on an edge (e.g., is at 
a local maximum of the gradient magnitude) , and R is a given 
distance 6 away from Q in the gradient direction. In the 
matrix P] defined in this way, diagonal conce.itration still 
correspond:* to coarseness, since if 5 is small relative to the 
texture patch size, R should be interior to the patch on the 
edge of which Q lies. However, may be more sensitive to 
coarseness changes than the P^ matrices were, since Pg is not 
influenced by point pairs that are both interior to patches. 




Another way of obtaining information about the spatial 
errangecient of the gray levels in f is to compute statistics 
of various local property values f measured at the points 
of f. 

As an illustration of how local properties can be used 
for texture description, let £ = (Ax, Ay) be a displacement, 
let fg(x,y) = f (x,y) -f (x+Ax, y+Ay) , and let p^ be the histo- 
gram of fg. Suppose that f is composed of patches of size s. 

If ^ is short relative t > s, the high entries in p^ will be 
concentrated near 0, since pairs of points £ apart will usually 
have small differences in value; but if is large, the entries 
in Pj will be more spread out. (Note, in fact, that Pj(z) 
is the sum of the entries in the matrix along the line 
parallel to its main diagonal for which i-j * z.) Thus the 
concentration of p^ near 0 is a measure of the t"coarseness" of 
f relative to 6, or equivalently, the spread of p^ away from 0 
is a measure of the "busyness" of f. Here again, these pro- 
perties may depend on direction. Similar remar)cs apply if we 
use absolute rather than signed differences; this simply folds 
p. over on itself at the origin. 
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Th« gray level (absolute) difference histograms are 
not affected by shifting the grayscale (as cooccurrence 
matrices are ) , but they are affected by stretching it; thus 
they too should be used in conjunction with grayscale normali- 
zation. Various statistics can be used to describe p^, includ- 
ing its mean (^ £zp^(z), if we use absolute differences), its 
second moment (£z p^(z); this is proportional to the "contrast" 
statistic for the corresponding cooccurrence matrix ) , its 
entropy (-£p^(z)log p^(z)), and so on. 

A wide variety of local properties f can be used in place 
of fg for texture description. For example, we can use com- 
binations of differences, such as the gradient (magnitude) 
or Laplacian; matches to local templates, such as spot, line, 
corner, or line end detectors; and so on. f can be a predi- 
cate, e.g. 1 if an aibove-tlireshold difference is present and 
0 otherwise; in this case, the histogram consists of only two 
values, and its mean tells us how many edges (or spots, lines, 
etc.) are present in f per unit area. Another possibility 
is to count local gray level maxima and minima in f; evidently, 
both the number of edges and the number of extrema per unit 
area are measures of "busyness". More generally, we can count 
occurrences of arbitrary local patterns of values in f. 

We can use second order as well as first order local 
property statistics as texture descriptors, by constructing 
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cooccurrence matrices of the values cf £' in given relative 
positions. If desired, we can use only selected pairs of 
points in constructing tiie matricea, e.g., pairs of extrema 
or pairs of above-threshold edge points, and we can use dis- 
placements at each point that depend on f , e.g., displacements 
in the gradient direction, as at the end of subsection (a) . 

Rather than using a set of local properties, e.g., dif- 
ferences computed for a set of displacements, we can use a 
single property and measure it for pictures derived from the 
original one by a set of local operations. For e.''ample, sup- 
pose that we use a sequence of local min (or max) operations, 
and at each step, measure the average gray level; the rate 
at which this decreases (or increases) is a measure of the 
coarseness of the high-valued (low-valued) patches in f. For 
a binary-valued f, the analogous idea is to shrin)c (or expand) 
the I's in f repeatedly, and at each step, count the number 
of I's. This approach, using generalized shrin)cing and ex- 
panding operations, has been extensively 

used for textura analysis in microscopy. 


In the previous two subsections we saw how various 
statistics of the cooccurrence matrix, difference histogram, 
etc. for a given displacement ^ = (Ax, Ay) provide useful in- 
formation about a texture. Thus the set of values of a given 
statistic a as a function of ^ (in particular, for relatively 
short 5 's) can be used as a texture descriptor. 

For example, let fj = f (x,y) f (x+Ax, y+Ay) , and let a be 
the mean of fj; then a as a function of ^ is just the autocor- 
relation Rj, i.e., the expected value of the product of the 
gray levels of a pair of points ^ apart. By the Cauchy-Schwart 2 
inequality, this takes on its maximum value for ^ = (0,0). 


(Proof: y^-Ay) factors 

[Zf (x,y) ^Ef (x+Ax, y+Ay)'^)-^'' 

in the denominator are the s 2 une, so that the denominator is 

2 

equal co Zf(x,y) , which is R^(0,0).) The rate at which 
falls off as £ moves av^y from (0,0) is a measure of the 
coarseness of f; the i:_iloff is slower for a coarse texture, 
and faster for a busy one. 

Similarly, let f^ = I f (x,y) -f (x+Ax, y+Ay)l^, and let o=v 
be its mean, i.e., the expected squared gray level difference 
at two points ^ apart; this . descriptor is sometimes called 
the varioqram of f. (Compare the use of the mean of the 
absolute difference histogram as a texture descriptor in 
subsection (c).) The rate at which its value rises as 6 moves 




81 


i; 

i; 

I 

i: 

i; 

i: 

i; 

i; 

i: 

r 

i: 


i; 

i; 

i; 

i: 

i; 


away from (0,0) is a measure of the coarseness of f; the 
rise is slow for a coarse texture and fast for a busy one. 

Note that v(4.) = E{ [f (x,y) -f (x+Ax, y+Ay)l^} = E{f^(x,y)> + 

E{f^(x+Ax, y+Ay) } - 2E{f (x,y) f (x+Ax, y+Ay)}; here the first 
two terms are just R^(0,0) and the third is -2R^{^), so that 
v(M = 2 (Rj (0,0) -Rj (M ) . If f is isotropic, the values of R^ 
and V depend only on the length of not on its direction, 
so that they become functions of a single variable. 

A texture can be modeled as a correlated random field, 
e.g., as an array of independent identically distributed 
random variables, to which a filtering operator has been ap- 
plied. This model suggests that a texture can be described 
by its autocorrelation and by the probability density of the 
original random variables; the latter can be approximated by 
a histogram after a "whitening" operation has been applied to 
decorrelate the texture. If we use the gradient or Laplacian 
as an approximate whitening operation, the histogram is just 
a histogram of difference values, as in the preceding sub- 
section 

The Fourier power spectrum |f|^ 
and the autocorrelation R^ are Fourier transforms of each other. 

2 

Thus |f| can also be used as a texture descriptor. The rate 
2 

at which |f| falls off as the spatial frequency (u,v) moves 
away from (0,0) is again a measure of the coarseness of f; 
the falloff is faster for a coarse texture and slower for a 



1 

f 

i 
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busy one, since fine detail gives rise to more power at high 
spatial frequencies. Samples of |Fk taken over rings centered 
at (0,0), or over sectors emanating from (0,0) (to detect 
directional biases), have often been used as texture features. 


Other transforms of f can also be used as a source of 
texture features. In practice, features based on |f| (or R^) 
seem to be somewhat less effective for texture discrimination 
than features based on second-order or local property statistics. 
At the same time, computation of |F| is more costly than 
computation of a few statistical features (recall that they 
usually need only be computed for a few ^*s), unless we compute 
it optically. 
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e) Region-based descriptions 

The texture descriptors considered so far are derived from 
local or point pair properties. We conclude by briefly dis- 
cussing texture description in terms of homogeneous patches 
or "primitive" regions. Several types of texture models are 
based on such decompositions into regions. For exeimple, 
textures can be generated by using a random geometric process 
to tessellate the plane into cells, or to drop objects onto the 
plane, and then selecting gray levels (or gray level probabi- 
lity ii’nsities) for the cells or objects in accordance with 
some probability law. 

If we can explicitly extract a reasonable set of primitives 
from f, we can describe the texture of f using statistics of 
properties of these primitives — e.g., the mean or standard 
deviation of their average gray level, area, perimeter, 
orientation (of principal axis) , eccentricity, etc. Second- 
order statistics can also be used — i.e,, we can construct 
matrices for pairs of values of the area (etc.) at pairs of 
neighboring primitives (perhaps in directions defined by each 
primitive's orientation. Of course, this approach depends on 
being able to extract a good set of primitives from f at a 
reasonable computational cost. A related, but much simpler, 
idea is to extract maximal homogeneous blocks (e.g., runs of 
constant gray level in various directions) from f, and describe 
f in terms of (first or second order) statistics of the block 
sizes (e.g., run lengths). 
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In general, the description of textures in terms of 
primitives may be hierzurchical; the primitives may be com- 
posed of subprimitives, etc., or they may be arranged into 
groupings which in turn form larger groupings, etc. This 
makes it possible to define placement rules for the primitives 
in the form of stochastic grammars. Texture analysis can 
thus be carried out, in principle, by parsing with respect 
to a set of such grammars. 
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SHAFK IDENTIFICATION USING 3-D FEATURES 


C.M. Bjorklund 
H.S. Loe 

Lockheed Research Labs 
Palo Alto, CA 943U4 

Image analysis continues to pose significant difficul- 
ties for automatic computer analysis due to ths unpredicta- 
bility of object signatures, variability of scene and il- 
lumination content and information lost using two- 
dimensional imagery. Systems providing both intensity and 
range information permit geometric analysis of the scene to 
be performed concurrently v/ith grey scale analysis. Laser 
range imagery provides this capability. Pixel values in a 
range image measure the distance to the nearest surface 
along the ray; thus, piiysical measurements of shapes can be 
extracted. Two applications will be described. In the 
first, pi anar surfaces are iHentified and extracted for 
matching in scenes containing buildings [1]. in the second, 
vehicles (e.g. trucks and tanks) are discriminated based on 
features extracted from the 3-D data. 

D.L. Milgram and C.M. Bjorklund, "Range Image Process- 
ing; Planar Surface Extraction", 5th ICPR , Dec. 1930, 
pp. 912-919. 
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Automatic Photointerpretation via Texture and Morphology Analysis 


Julius T. Tou 

Center for Information Research 
University of Florida 
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Abstract 

This paper discusses computer-based techniques for automatic photo- 
interpretation based upon information derived from texture and morphology 
analysis of images. By automatic photointerpretation, we mean the deter- 
mination of semantic descriptions of the content of the images by computer. 
Such descriptions include a narrative report identifying the objects in 
the image and describing their characteristics and relationships. Our 
approaches consist of two major tasks: (1) Morphology analysis, and (2) 
textural analysis. Morphology and shape information enables us to make a 
preliminary identification of the objects or regions. In both tasks, a 
growing knowledge-base is generated from past experience and a priori 
information. Objects with distinctly different morphology and shape are 
recognized and these contents in an image are interpreted. To make a 
finer identification and more accurate interpretation, we make use of 
textural information. 

To perform semantic analysis of morphology, we have developed an 

heirarchical structure of knowledge representation. The simplest elements 

in a morphology are "strokes", which are used to form "alphabets". Tlte 

"alphabets" are the elements for generating "words", which are used to 

describe the function or property of an object or a region. The "words" 

are the elements for constructing "sentences", which are used for semantic 

description of the content of the image. We realize that in many cases 

morphology alone may not be sufficient to make positive identification and 

accurate interpretation. Photointerpretation based upon morphology is 

then augmented by textural information. q, 

f ^51: J Ly .. .■’« I u . ■ , utiKii 
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To perform textural analysis, we make use of the pixel -vector approach. 
Each pixel or cluster of pixels is represented by a property vector which 
characterizes the pixels belonging to an object or a re*gion in an image. 
Pixels of similar properties are extracted by a correlation and clustering 
technique. Since an object or a region may contain several types of 
pixels, an object may be decomposed into several clusters of pixels with 
different types and properties. The features of the decomposed objects are 
used in automatic photointerpretation. When objects can be decomposed into 
similar clusters, we determine the textural rythm as positive identifica- 
tion of the object or the region in addition to morphology information. 

The knowledge-base Is augmented with acquired information from image ana- 
lysis. 

Some experimental results of our knowledge-based photointerpretation 
system will be discussed. 
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FLIR TARGET SCREENING 
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OF POOR QUALITY Raj Aggarwal 

Systems & Research Center 
Honeywell Incorporated 
2600 Ridgway Parkway 
MinneapoliSr Minnesota 55413 


Summary 

The remote sensing applications typically involve a sensor that 
acquires data from the phyiscal world, a processor to process this 
data and most likely a controller that performs certain mission 
related functions. Remote sensing is characterized by lack of 
control over the sensing environment and the scene being sensed. 
Thus, in addition to the problems posed by the recognition task 
itself, there are special problems due to uncontrolled environmental 
factors, including noise and coherent clutter, and in some cases, 
uncooperativeness on the part of the objects that are to be 
recognized (countermeasures, camouflage). 

One application of remote sensing has been in the area of FLIR 
target recognition. The sophistication of reconnaissance and strike 
systems is constantly increasing due to the high threat operational 
environment. Thus, advanced forward looking infrared sensors are 
integrated on high performance aircraft. The fast loading and high 
information rate of advanced sensors has made it imposssible for a 
human to perform the target search/deduction/recognition task 
accurately, consistently, and in real time. A lot of work has been 
done by university/industry teams towards the development of FLIR 
target screening technology. 

A typical target screener would consist of segmentation, detection, 
and recognition stages. Segmentation step would typically involve 
the ability to locate the regions of interest. The detection stage 
is to separate out the clutter from potential targets and the 
recognition step is to label the type of the target. The process to 
date includes not only the simulation of this technology in various 
laboratories around the country but also the development of real 
time hardware. Some of the hardware boxes have been tested in real 
time in helicopters, etc. 

In FLIR much work has been done on the segmentation and recognition 
of invidual targets, some of it using structural as well as 
statistical methods. One key problem that still remains is the 
ability of these techniques to work under varied conditions without 
extensive retraining of the algorithms. This deficiency is what 
leads us to the development of what are known as multi.-scenario 
target screeners. There the key ability for the screener is to 
learn from what it sees through its sensor and adapt 
accordingly without operator intervention. The work being done by 
Honeywell in this direction will be discussed at the conference. 
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Structural Analysis Techniques for Remote Sensing 


Linda G. Shapiro 
Department of Computer Science 
Virginia folytechnic Institute and State University 
Blacksburg, Virginia 24061 


SUMMARY 


Structural analysis uses knowledge of the properties of an entity, 
its parts and their relationships, and the relationships in which it 
participates at a higher level to locate and recognize objects in a 
visual scene. For exanple, Bajcsy and Tavakoli [1] used spectral and 
shape properties of roads along with knowledge about required 
connections to other roads in a system for computer recognition of 
roads from satellite images. Tenenbaun, et. al. [7] set up g&xnetric 
correspondenca' between sensed images and symbolic reference maps to 
aid in monitoring or tracking predefined targets. We will discuss the 
basic techniques required for structural analysis. 


Che problem is the representation of structural knowledge. 
Production systems and relational descriptions are the two major 
classes of representation used so far. Production systems include 
picture granmars [2] that use p>roduction rules to parse pictures and 
expert systems [4] where knowledge about pictures can be stored and 
retrieved. Relational descriptions include trees, graphs, n-ary 
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relations, complex relational structures, semantic nets [ 4 ], and 
fVames [ 3 ]. 
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A second problem is the development of efficient algorithms for 
using the structural information to help analyze an image. In [5] v#e 
defined a general model of a structural description and compared 
several inexact matching algorithms for finding the corresponder.ee 
between models and images. A simple scheme called forward checking 
showed the most promise. Parallel hardware can also be used to speed 
up the matching process. In general, the more the problem can be 
constrained by knowledge, the faster the matching can be done. 

A third problem consists of techniques for storage and retrieval of 
relational models. A knowledge database will typically consist of a 
large number of models and/or frames. Matching an unknown image 
against all of them is impossible. Thus schemes for organizing the 
database of models for fast retrieval of those models most appropriate 
to a given image are essential. In [ 6 ] we discuss some preliminary 
schemes for database organization. The problem of organizing 
structural knowledge and knowledge in general is an important topic of 
current research. 


ORIGINAL PAGE IS 
OF POOR QUALITY 


REFERENCES 


ORIGINAL PAGE IS 
OF POOR QUALITY 


93 


1. Bajcsy, R. and M. Tavakoll, "Computer Recognition of Roads from 
Satellite Pictures", Department of Computer and Information 
Science, University of Pennsylvania, 1974. 

2. FU, K.S. , Syntactic Methods in Pattern Recognition , Academic 
Press, New York, 1974. 

3. Minsky, M., "A Framework for Representing Knowledge", in The 
Psychology of Computer Vision , P. Winston, Ed., McQ*aw^ill, 19757 

4. Nilsson, N. J., Principles of Artificial Intelligence , Tioga 
Publishing u.., I960. 


5. Shapiro, L. G. and R. M. Harallck, "Structural Descriptions and 
Inexact Matching", IEEE Trans. PAMI, Vol. PAMI-3, No. 5, 1981. 

6. Shapiro, L. G. and R. M. Haralick, "Organization of Relational 
Models for Scene Analysis", TR CS81011-R, Department of Comr iter 
Science, Virginia Tech, 1981. 

7. Tenenbaim, J. M., H. G. Barrow, R. C. Bolles, M. A. Fischler, and 
H. C. Wolf, "Map-Guided Interpretation of Remotely-Sensed 
Imagery", Proceedings of IEEE PRIP, 1979# PP* 610-617. 





Linda G. Shapiro 


37AOCTOrt^L HA)OUJLCOC€ 
CP f\N joclu</e& 

A. rtf)ouJ/€^€- of ifs 

a,. of 

a<i d fheih MJlo.'ti on «r4//o^ 

3. Knotj/ffi^e 0 / 
ttlcct/onsJii^^ fh tJhi^ it 
/>a<‘’fic#y<vfe4 A h*Ahtn 

/<vcl 

ORIGINAL PAGE IS 
OF POOR QUALITY 



ORIGINAL PAGE 19 
OF POOR QUALITY 


^0ex> MeTtOO/C.K 

UUAAJ 66T7IN^ 


/j itrx 


/f‘S />)/)•# 

rcjfor )4 arr^riAfd ih a an 4 


it Arc 
iAro4 A^ry 




4 




Hem) ca,f\ iuc 
>r//*(/cyu/»-vi HnouJ/ed^^ iro 
a wey 0*0 Htmt ft can 6e 
(K^di cuf ujho<if, 

Joh ii /j A antk/yz9 


ORIGINAL PAGE IS 
OF POOR QUALITY 


•"¥ 



1 


ORIGfNAL PAGE I9 
OF POOR QUALITY 


97 


CS7/VTACTIC j)£schipt/(5/VS 

s-^ ('> 

c 




5 


s s 


i; 

i; 

r fjELAT/o/v^L o£5oz\mo^s 



ORIGINAL PAGE IS 
OF POOR QUALITY 


98 


REPRtSfNT/hfG 

MNcijii:r>c£: 

A €c o T p^lfl 'fPES 


piauRE mhtiAfs 


£ y.fe.nsicn of phw’e 

* ORff 

OSZCsC f n't erf nor* rr-^ 


20 


dnfo 


I 


ORIGINAL PAGE IS 
OF POOR QUALITY 


SAMT! i 


Pffc nueTtc N. 









ORIGINAL PAGE IS 
OF POOR QUALITY 


OERIVATICM 

Ric- /.IT r 


rp 




mm 


2. 


mm 




X 


#■/ 1 f 


H I I 


mm 


fi\B 


i^lJ It. 


n \e \R 
























4 


‘ 101 

I ORIGflMAL PAGE IS 

OF POOR QUALITY 

i: 

[ ANOTHER TRtANUB 

I 

[ (.triangle (r y 2.) 

I (CVERTEy %) (VERTE'k X) 
i: (VERTEX 2) C£iS X /) 

f (BIS X z) (BIS y z) 

I (NON COLL X y z) 

I C ( yERTtces 

r (LIST X y 2 )))) 

i: 

i; 

i; I 


102 


ORIGINAL PAGE IS 
OF POOR QUALITY 


f-fOL. OC '/CU PARK£ a 
PicTORe': 

MBTHOOl: CREATE A 

STR/AfO OESC R! RtNC- 

rue P/CTORE ANO PARrg /f. 


fIBWOO 2 : USE TVE 

GRAMMAR Tc niilt^E tP£ 


EE arc R 


IN 


RElAT/CA/SR>r’^ /A! THE 
PtfTl/E£ 




ORIGINAL PAGE IS 
OF POOR QUALITY 


'fiELATIONAL STKUCnltR 

1 * 





(D 


/O 


PlCTi^R £ 


/?ffLAT/tflY5| 




C’. 




-tio 






TH€ A^/^TcM/A/6. 

ORIGINAL PAGE IS 
OP POOR QUALITY 

I (xivefu ^ff/*ocfu^a/ mocf< I 

0-f an cnrf»fy , 

D I? a. Aiv^ri 

€0+1 fy hcus jorne 

c/utrAcfen *//»€ moo^e/., 

Of li*) Aocf oo tnH'fy tiL 

jivtn iivK^e iu/fAi jAm^ 

j/irocforo,/ ciiarncferiefCcs cu 

mod^ , I 

oi' A) d^rmint, if oo etlii'ro, 

1 "* 

/ifioje jafut/cs ccosh^ihi^ 

jp€ci 4 / Ay mod^l' ' i 

■ _J 


ORIGINAL PAGS IS 
OF POOR QUALITY 

EXACT AAATCH/AIG- 

• . I 

StrUctorO-l Descript/on of an OijecL^ 

D- (P.fl'), 

P ■ £ . 5 3 I'-S a- set 

of P/? i A1 1 r.' l^ES . 

Pi, “ /^#T’iiu/es V«.lues^ 

R. “ f • 5 

of r\o.med reletfi^^^S ov£r f 





A//?j, /s 0, /V/)Aie. -Aft 

Pa ^ P ^ fe^ ^e»o?e 

/s. a^^ A1^-a.rj reldf/'o/? 



ORIGINAL PAGE IS 
OF POOR QUALITY 


EXACT 

match I/VQ. 


CArieti Prlrtii+ive 

mrcites 



Prototype Pri'mifiVe, p. 

If 


107 


ORIGiNAL page is 
OF POOR QUALITY 



4 0 

5 D 



Flgucc I 1 illustrates the coaposltlon of 

binary relation R with napping h. 

Composifion of ou 
(With oc 


Let R ^ ^di h- P“> Q 



k//fA At'/Pi')' Si 


,\ 


OF POOR 


qvjauTt 







k: 1 A 

2 - B 

3 - C 

4 -► D 

5 B 

nek c S 


• ' »u“Ci « M-n " 

rclstloD S. 


f\ (?ez.AT/OftfA4- 

^ror. ^ ^ 

u . p ^ Q 5A-f ) j -Ty /« i. R* ^ “ 



109 


OftiGlNAL PACE F5 
OF POOR QUALITY 

either MATCH oeF/zV/r/oA/ 

Le-t Df “ Cf,R') he. a. iirft^-lype. 

sfruc+Ur'^.l descripTiorx 

P = CPo P^i 

i^-b Dc"CQ,S') O- c<siod.‘cl<i+e 

^yruc.fu»^<»./ o/escr) pf/of^ 

Q * 1 Q«>- - > Q '»»»1 

S ‘ Fr/l^S., 5,"), ...j (/4'5 k,^0] 

Oc. MATCHES Dp if +h^«. IS 
a. h = P -» Q s-^+UTy/no 

D h (Fx') -*0)1*=^ ~ ^4 

* A/Sx H 



ORIGINAL ^ 

OF POOR QUALITY 


110 



/^ATCrt/>U 6 - 

UJeA^h+edl Pco4t>fN^-e. ■S-ft'oc-|'vjro.\ 
De. ic.r i pr/oH' 

0 = ^ P 3 '^P > 3 ''^a. ^ 



%. iWpf4*i 


Wtt ■“ { Wi , • •• 1 ^ 

U 3 ^: £o,ll 

^ vaJi (-«-■) •=•:!. fw u A*' K 

_ _ _ 

l;ij Cfl t-)w. (pf.Vtli -Ku*. toVl^h+in<j -Rjncfior) 

lijfi K , ^ ^ /(/-/up/«- 

(A-’e 4i O') /"y nof /oflS 



Ill 


ORIGINAL PAGE 13 
OF POOR QUALITY 

€»* />omomo^f /)km4 

Ac i ^ 

S 2 Q 


f\ n 

io 

IS (X 


C • hom^mor^hi^m -P pom 

S uuith respedii v/j 

P*^ ^cch)^f^y in^ 


y" 

R, 


i- e 


Aoh. 

fht «-F 


ORIGINAL PAGE jS 
OF POOR QUALITY 


/AISXACT MATCH DeFM^/T/©>V 

Ut Dp te ^ \A/6| pre+af'y^e 

S'Kvtc+urAl desc <r pi- } 0Y\, 

Dp ’ (TB ujp, Rp , VUBp') 

P' £Pi, ifi,! 

W/?P ^ 

Lsit: Dc ^ C€tncJi 

y^jyu (/r <3L / /iifescr fp ^ 

c, hcD 

C * 1. ^1^ .■; C/w3 p f^~ 


I 


ORIGINAL PACE 15 
OF POOR QUALITY 


Oc. /hcnact/y mednhes Dp 

(jurfb reapec.'t *f'o 

at'friky'K. Value fhre^holJs ~ 

rhisi.,/)^ Ciarfs H)r&sht!A aU 

r&J«L‘^/on E f6- I ff? . 6 R3 

if tiher^ /3 

h ' P C. U i null \ 

f) C: ^ Cj /'ntHcicfly rr\^i(ihfS 

PjC uj'ri^ r^sf^ ct ~Y. 

a) ^ vjp(Pi) ^ 

f O ' no II 

sD A/R^ - D/5^ => /) /i OA, 

homoroorphts)^ kjjtt l^i I^Z 'f'^ ' *' 



1 


ORIGINW. WOE © 
OF POOR QUAUTir 


114 


I i 


flHOthO> £ - homoir>Ofp^‘*or) 5 


7r*«' search uj'i+h- 


^ BACKT/^ AC.K\A)a 

Fo«.uiARP elfECKlAlG- 

e) 1,0 0 KA head OA/6 

.i 



ir 





1 

ORIGINAL PAGc Fa 115 1 

OF POOR QUALITY 1 

8ACK TMCK I A) G- 

J^n Shan't (pairs | j 

('Pi.Qi') jl 

on-h) the. error of -f^e. ] 

I 1 

par + )a\ >.(Y>cippin^ G.on^+ruc,+9e( | j 

so -par i\s +hcvn ^ . 


ORIGINAL PAGE IS 
OF POOR QUALITY 


116 


foRuDFiRD 

bacAfCAcKirio^ y 6UT»^» 

Keep a future Brror Tibbie. 



f*, h ltd'll e^vsc, on th^ 

pa.fh»L\ thH«.ntlo.M -rar. 

U') ^ ii>^ errer fK 

rovt ji> 6*P p^-T' 


I 

r 

r 

r 

r 

[ 

[ 

i 

I 

I 

I 

r 

[ 

i: 

i: 

r 


ORtGINAL page 13 
OF POOR QUALITY 

phoccdurc Fo^ op^vA/c^ r.e*T. 

“for 6 iha»ryj r^l«t+/or) 

an4 iiaa#y r>ela>idr\ S ^ (Ufh 

P r ocedore 0 Qc , Pasterr^ 

Sum. of- ^rrofi •- O 

■for eac./\ oolos'^an’fiafecS pnV»3i'f»V®^ otft 

R ih 

■por eacJ) Qj OH- 'ft>r g(o 

(f^c, P*) e « aal ^Qc^Op ^ S . 

fh«n F.£.T.U,^')-F.e.T.(z,i)-^*i(KA 

eni 

r^lAiSAfVCO • = /)€u/ /rttA)/morn /h 

“ 5om-.o‘P-^fyoi!S ^ 

if Sow. o-f. £‘<'r'ori Parkrr > 6- 

H)^f \ ■Po.i 1 r^fvrn 
€YNd . 

f^’huff) ( 5om — o*f- j 


ORIGINAL PAGf ® 
OF POOR QUALFPf 


tft cnecKi4;ft 

Before iiOS'hxrt'h/'a.tiV)^ 56 we. p«'> 

C^c,Q<T) 

i^chficK t-f 

error o-P peurfiM UbeWrt^ -^ar 

-h F.£.T o ,, 

+ Sowi o-f o-ll 

pr‘nwiri\/eS ]X 

\S> "HlftM 61^ 'fo ^ 

caU OPO/Vre. to 

Ay^<^ fAturn fkeuj fwturs erf or 5uw 

(ey.r\i. bo(s:h)y to F4<0 


t 


119 


! tooKA/fe^D (L1 oA/e 

LiK^ "foruJOLrol CLh?cJ<in« &U“F 
^ Cl(so jiD^ke.. lh'^^ CLccouAt 

I* 

€jTCor can c^iouT 

1 1 

j cio e. “to ^on>5 y^“t 

C ( flo, 

[ 

I ORIGiNAL PAGi rs 

OF POCn quality 

I 

I 

I 

I 

I 

I 

I 

[ 






C€tn A/« otA^m'z^^ 
Cl ^^*f€ oy^ 

sSitvciur^ mod^U fon 
/Ajf af iJ)9S^ 

ffiod^S moJ t t cif ^ 

for use, tb AA4//^//?j cu 
^ireo ffba^e^ 


0<iu^ 


original page « 
OF POOR QUALITY 


/Iff ro ach 

0 ou rzlocffono^\ 

measure Mr./ ts 

p(^sy s o R i3*morpfcic "K S 
D(f?,5) ^ DTR.T) tDCr^S) 

all r€la'i/*^4t^ /noc/Oi 

O (/J/iy f^^^ono^f clt shines 
/»€40fo^C, c/c/j^e/' fHodl^U 

i'oi^ j/V/a/' />io(fe./j 


0..:C;iNAL PAGE fS 
OF POOR QUALITY 


j') >Sc/ect 'for ^qOl C 

Ct We /^c . 

rcpretcfvfafiv^ 

6 tit Model 5 *r4cff«sfy/nj 

S' — o»»Vi ^ o(/?, ^0 

/^«C R'cC <^<C 

y) /^a^c/l unknown eniif/ej 
/© Me c/oj^r r^prej^nMii/cs 




fkoS^ C/0J.^C/\J 

rc/orej € hfcj-m s h<^ k p rcpisin^t 


124 


ORIGINAL 

OF POOR QUALITY. 


ftt btsf /^u/fi , th^ 
c/ashrmj ^ndl 

r^pr«f«nf<Lf/ife ts^^cftotv 

ithoo/ct j*€i^Myy 


for eadj Uojf^j' C. /fj 

re^refeAfA.f/1/R. ^ h 

CL fhf^^shoid Tc. fb<\i 


0 € C. ^ ”T^ 

oO Kf^ ^ ^ ^ CRf Rc) > Te 


ORIGINAL PAGE IS 
or POOR QUALmr 


125 


F 





I 


WMr A 50 OT MfX/tCT At 4 TCHM«^ 

1 


UJM if onHoow^ enii^ U ! 

Oof ijenfical io AAy mecf^f fh 

fh^ /s cJc^iesi; /« 

«* 

4 om^ md^l M ih C 

r^e^es^nMiv^ /(c w fhrejM Tc. I 

Dru, 8 c'^ " 0 (' 0 ,A 4 > + 0 r/vi^^cV' 

^ Tc. 

X/ ^(Uj/A) ^ f^CFt 
0(Uj /ii) Xc. f ^ , 


! Xv pf«cH<jttUfe c»f! cAo 6^^ 4^ € «»d( 

,. Mc^€Oi€- €«C0 fhtesSo/^ Ay & 30 

I wf 4 )ifhm d or a /»cFo^/ 

I 4i»^// r94tlh fht f^^r€i€nM f(jc cfds 

» 


126 


/^e/v\oTB p-^c/sce*/'^ 

) 

\ 

k 

Givtn: LhfiO^^T /Maje of cl. 

fnourtfciinov)4 arco^ 

PcobItiK* Aufom^JtiCA/zy ld'Oi\\ify 

ol(rtc'fro*c dp P^om. 

77^/i /\s ^€,ihy 4 ch* (![ 

at vJifh the. of 

JTf^cru^AL Kno\j} L^OOB . 


ORIGINAL ^ 

OF POOR QUALITY 


•• an CLlr€mcfy 

I thh 

i: 

I KiJoiuLecfc 

no 

i; 

14 ) Jufl }//o/>l ffmt /CPt fi 
^ €a j t "ho u c j i 



ORIGINAL PAQf 
OF POOR QUALITY 




^ioS‘6^ 6o(joc/A/*y f^tit is yfl/*>jr 
on fhn. /ef't nncf 
on fil^ rkht 

4- ^UA/ 

V^LL€i ’ bcufltfory fh^.f is 

on fht. h'ft o*fci 
{^otK on fhe. ni\ht 




129 


eie\/A7/0A/ 


MOOeL 


n 

i > 

ORIGIMAL PAG? ISf ! I 

OF POOR QUALITY I 


^fart ioifh KnovJn fioihfs 
0-P /ouiwfc e(tv/A.f/on «n4 

AfpvJ 



\U9 ^fcxu (kIoa^ v«.r'c/ 

a.nd oo+»*>Arb| 



OWOINW. PWE B 130 

Of POOR 

HoUlZOifTAL 



ou 


Cl k 

^ X - 


OL /owei* li 

X * r»d^€ 



ORIGINAL PAGE fS 
OF POOR QUALITY 


i . 

i; 



• j(//?cVvo/f 6^’ 

'P<ior 

bo(jfido.ry 

J64 mini’s 

u*f tb 

mort. 

ticfo\ 

es tAon. 

• 

¥A 

V 

(kr>d 

kj'tth 

ff*€. of 

/^5 /cn^fst 



fhu.^ 


fhreshtU 





132 

L. H/tf' «Ai Lt:or>.€: 


r 

t 

k- 

[ 

I 

f 

I 


AN(.LC3 

ih&l 

3^'S3 

UP 

OotfJU 

s/iO 

<1© 

>10 

SI, S3. 

^3 

s /fo 

> 10 

<^0 

^Ji,ss 

^ / 

^IfO 

• 

> 10 

>10 

s s ^ 

1 

• 

• 


ORIGINAL PAGE IS 
OF POOR QUALITY 






134 


^S83 15773 

DETERMINING 3-D MOTIGN AND STRUCTURE 
FROM IMAGE SEQUENCES* 



T. S. Hu«ng 

Coordinated Science Laboratory 
University of Illinois at Urbana-Champaign 
1101 West Springfield Avenue 
Urbans, Illinois 61801 


Summary 

The determination of 3-D motion and structure from image sequences has 
many applications, including inter frame TV coding, target tracking, and 
robot trajectory planning using visual feedbacks Past work in the area led 
to results which involved the Iterative solution of nonlinear equations [r-3] , 
and the questions of convergence and uniqueness %»re not resolvi^.d. In this 
talk, we shall present a new method [4] of determining 3-D motion and struc- 
ture from two image frames. This method requires eight point correspon- 
dences between the two frames, from idiich 3-D motion and strilCture parameters 
are determined by solving a set of eight linear equations and a singular value 
decoaq>osition of a 3X3 matrix. We also show that the solution thus obtained 
is unique. 
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where (Ax,Ay,Az) is Che amcu C of translation, and R is a rotation matrix 

[ 1 . 6 ] 
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where n^, n^ and n^ are the directional cosines of Che axis of rotation 
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and 3 is the amount of rotation between the two frames. 
Assume the amount of rotation, d,is small. Then 
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And we have, using Eqs. (1.22) and (1.15) and after some algebraic manipula- 
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RE: OAB5-39 


May 10, 1982 


Dr. Larry Guseman 
Department of Mathematics 
Texas A&M University 
College Station, Texas 77843 

Dear Dr. Guseman: 
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I would like to thank you for Inviting me to the NASA Workshop on Image 
Analysis. Since I work in the area of technology transfer and applications 
of remote sensing, I viewed the conference from the perspective of how 
fundamental research influences applied research. 

In the area of Image segmentation, we now have many tools for clustering and 
classification. The extension to be made from cu: rent capabilities will 
perhaps make use of ancillary data for classification refinement via digitized 
soils data, terrain illumination correction, etc. With many layers in a data 
base, total registration accuracy is a weak point, as well as mechanism to 
quickly process mixtures of raster and polygonal formats. 

Shape and texture analysis seems to me to present the most exciting possi- 
bilities - particularly given the finer resolution of Landsat D. Since we 
know a human analyst can identify features from shape and texture, perhaps 
images can be enhanced to iteratively improve the interpretablllty until 
ultimately some features can be machine-recognized. The role of color in 
this process has been relatively untapped which to me implies that the exten- 
sion of algorithms from 2-D to N-D is not at all trivial and deserves a great 
deal of study. 

The area of structural knowledge relates to our needs in that we often model 
a desired variable (irrigability, exploration potential, grazing capacity) as 
a function of other variables. A structure within which we can analyze and 
weight the variables, and arrangements of the variables, would be beneficial. 

As to where to look for the evolution of new image processing canabllltles, 

I personally have been dissatisfied with statistical approaches. They seem 
to deal with Images in measurement space only and make assumptions that are 
not true often enough. The best forward strides I have seen are practical 
and arise from a well-defined problem. 

Again, I enjoyed meeting you, found the workshop stimulating, and am looking 
forward to visiting A&M again. 

Sincerely, 


' 

Susan K. Jenson 

Senior Applications Scientist 

Geoscience Section 


South Dakota Operations - EROS Data Center • Sioux Falls, South Dakota s7I9« • Telephone («os) s»4-«sii 
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May 13, 1982 

Dr. Larry Guseman 
Dept, of Mathematics 
Texas A&M University 
College Station, TX 77840 

Dear Larry, 

The following are some suggestions for future research 
areas under the NASA Fundamental Research Progreun in 
Pattern Recognition; 

1. Application of AI methodology to develop "expert 
systems" for various interpretation tasks. 

2. Automatic registration of multisensor data, and 
of images with maps. 

3. Study of VLSI architecture requirements for remote 
sensor data processing and analysis. 

4. Development of database management techniques 
applicable to remote sensor data. 

We hope these ideas will be useful. We enjoyed the meeting 
and look forward to seeing you again soon. 

Sincerely, 


Larry ^ Davis 
Associate Professor 



Azrjiel Rosenfeld 
Research Professor 
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